June 17, 2025 • 42 mins
Search is changing. Tried and true methods of old are no longer effective. In this episode, Don MacKinnon, CTO and Co-Founder of Searchcraft, joins us to discuss this rapidly changing area that has been disrupted by AI. Don shares his journey from the publishing industry to building a modern search solution, and discusses how AI is fundamentally changing content discovery. We explore why traditional SEO strategies are becoming less effective as Google search referrals decline, and how content creators need to rethink their discoverability strategies. Don also provides a fascinating look at the technical decisions behind Searchcraft, including their choice to build with Rust instead of Java or C++, and how they're positioning their product in an increasingly AI-dominated funding environment. Whether you're a developer looking for better search solutions or a content creator concerned about the future of discovery, this episode offers valuable perspectives on navigating the changing digital landscape.
Where to Find Don MacKinnon
- Website: https://donmackinnon.dev/
- LinkedIn: https://www.linkedin.com/in/dmackinnon/
- BlueSky: https://bsky.app/profile/donmackinnon.dev/
- Searchcraft: https://www.searchcraft.io/
Show Links
- Plausible Analytics: https://plausible.io/
- ClickHouse Database: https://clickhouse.com/
- Rust Programming Language: https://www.rust-lang.org/
- Elasticsearch (Comparison): https://www.elastic.co/
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Don (00:00):
has wide-reaching implications for content
creators.
Companies they're seeing theirreferral traffic just like
plummet off a cliff like betweenthe AI summaries and just them
Google driving referrals totheir own platforms like YouTube
.
Like the inbound traffic isreally diminished and so a lot
of companies now like they'rereally having to reevaluate like
their whole like informationdiscovery strategy.
William (00:34):
Welcome back to the Cloud Gambit, where we unravel
the state of cloud computingmarkets strategy, emerging
trends or whatever else.
To the list, I'm William andwith me, as always, is Yvonne.
Eyvonne (00:54):
How are you doing today , yvonne?
How's everything going inKentucky?
Good, the sun's shining heretoday.
It's warm outside, which is anice change.
It's taken until June for it toget warm and sunny, but it's a
warm, sunny day, and so that'sgood.
William (01:06):
Yeah, and you know what's great about it is it's
not humid.
So usually around this time ofyear, as soon as you hit June,
the humidity about kills you,and right now the humidity's low
.
It would be a great day to takea walk.
Eyvonne (01:21):
Yep, that is a bike ride is on my list as soon as we
wrap this thing up thisafternoon.
William (01:27):
Awesome.
So today we're diving into oneof the most, I would say,
fundamental aspects really oflike the modern web search, but
not just any search discussion.
We're talking about how theentire landscape is kind of
shifting beneath our feet andwhat that means for developers,
(01:48):
businesses alike, and our guesttoday is Don McKinnon,
co-founder and CTO ofSearchcraft.
How are you doing today, Don?
Don (01:57):
I'm doing great, great to meet you all.
I have to say, the reason Idon't live in the South is
because of the humidity, sothat's why I'm in Colorado.
William (02:08):
It's very smart.
Yeah, you have the dry sort ofthe dry climate.
You know the carry around lipbalm in your pocket climate, but
we've got the scorching.
You know if you walk outsideyou're sweating in a minute
climate.
So I would gladly tradeclimates for sure.
So I guess, starting out, don,let's start out with your story.
(02:30):
Take us through your journey.
Who you are and how did you endup co-founding a search company
?
What were the, I guess, thepivotal moments that led you
here?
Don (02:44):
Sure, let's see how far back do I go?
I would say you know, I callmyself a builder.
That's kind of like mybackground and strength.
I've been working in techprofessionally since 98.
My first job was at a startupduring the dot-com era.
Back then it was Pearl and PHPand and tickle was what I was
(03:07):
writing, um.
So.
But I kind of got my start onthe tech side, um, working in
media and publishing, um.
So back then in like the early2000s, late 90s, media and
publishing was like really bigum, those companies were huge
and they had like thousands ofemployees in like manhattan and
elsewhere like huge offices,right.
(03:28):
So I I did about 12 years at acompany called time inc and they
owned a bunch of brands peopleare familiar with, even if
they're not familiar with thatcompany called like time and
sports, illust, fortune, people,those brands, right.
So I actually started workingfor them from a subsidiary that
(03:49):
they had acquired in Boulder.
There was like a winter sportssubsidiary.
They bought them and at thetime I was one of the few people
in Colorado that had a lot ofexperience with Tickle, which is
a language you don't hear muchabout these days, but back then
the platform was called StoryServer, vignette Story Server,
(04:10):
and it ran much of the web backthen.
So they brought me in to belike okay, well, we acquired
this company, they're onColdFusion, we're going to
migrate from ColdFusion to StoryServer and we need someone
who's good at Tickle.
So they brought me in.
We need someone who's going totickle.
So they brought me in and Ikind of ran the dev team for
those sites.
(04:30):
So I did that for a bit andthen later you know publishing
industry they like to kind ofconsolidate and, you know, sell
each other their brands.
So they sold off the ones thatwere here in Boulder.
So I moved to New York and Iworked there for a bit on some
other brands.
I worked there for a bit onsome other brands and so that's
kind of when my role within theorganization shifted from just
being a developer to being likean engineering manager.
(04:51):
So I started running dev teamsand getting more involved in
like stakeholder conversations,working with product managers
and that whole thing.
And I worked for that companyfor about 12 years, um, but then
every pretty much every year Iworked there, they were doing
(05:11):
layoffs, um, and then one yearthey were like, okay, well,
we're gonna just cut your wholedepartment, so.
But it ended up being kind oflike a political situation in
that, um, somebody else at thedirector level that was the same
level as my boss wanted ourgroup, but they didn't want to
keep my boss, so they laideveryone off and then re-offered
us all our jobs, except for himthe next day to come back, and
(05:35):
so we're like well, you know,we're kind of loyal to our boss.
So we all said no and went off.
Um, so that's when I left thereand but that was kind of a
motivator for me to not work atthese big companies anymore.
Like you really have no controlover your kind of your destiny.
You know, if it doesn't matterhow hard you work, if you're
just like a number on a budget,like you could be cut right.
(05:58):
So I got into consulting.
At that time that was like 2013.
So first there's like justfreelance consulting back to
publishing industry people.
So, but not only back at time,but also condi, nast and other
publishers in new york.
Um, so I at that time I I wentoff on my own for a little bit
(06:21):
and then I joined a consultingagency that a guy I knew started
.
He was a PM that I had workedwith previously.
He's like hey, you know, we havethese big projects coming up
and we really need someone whokind of knows everything about
like you know how to stand up atech stack and architect and all
that.
So he brought me in and I ranthe engineering team for that
(06:43):
consulting company and weconsulted for companies like
Sony Music and Condé and Reutersand places like that, building
out CMSs, doing like datamigrations and then later
building out like a bunch oflike Elasticsearch
implementations, and so that'skind of where I got into
(07:15):
learning about search and likeit takes to deal with relevancy
issues and data indexing andcluster management things like
that.
So I did that agency for a bit.
And then I ended up joininganother agency with my current
co-founder of Searchcraft.
His name's Sharpie, so he'smore of a product and design
oriented person and I'm kind oflike the engineering guy, and
then we can kind of just combineforces and we did the
(07:35):
consulting thing for a bit.
But then on the side we werekind of building Searchcraft and
about a year ago we decided,okay, maybe we should wind down
the agency and try and do thisproduct thing full-time.
William (07:52):
Sounds like a journey, quite the journey.
That consulting work, grindingin the consulting space, is a
really good experience becauseyou come in usually as a
consultant, you come into whatsome might refer to as a hot
mess or just a reallycomplicated problem, and the
point being that that companymay not have the engineering
(08:12):
staff or the wherewithal tosolve or migrate or do something
.
You know that requires verydeep expertise.
So when you come into thosetypes of environments you don't,
and then everything's like oh,we need it done as fast as
possible when you you know wegotta meet those deadlines.
Don (08:28):
so yeah, that was kind of our niche, like we got known for
that um at our agency is, isthat, like a lot of
non-technical founders wouldbring us in because they have a
product vision but they didn'tknow what it meant to like scale
from zero and build an mvp andthey wanted somebody who had a
reputation for moving quicklyand that could handle everything
.
Eyvonne (08:48):
And even at some startups that we worked at where
they had technical people butthey couldn't move as fast as we
could, right, so they wouldcall us One of the things a lot
of folks don't understand who'venever been in consulting or
worked in and around it isreally the trick in consulting
work is being able to learn andthink on your feet.
(09:11):
Oftentimes, when you work for aconsultant, you may have a core
of experience, but very oftenyou get thrown into a situation
where you don't have the exactspecific expertise that you need
to solve the problem.
It's more about being able tounderstand what's going on,
pivot and then learn what youneed to do.
(09:33):
Learn what you need to knowquickly and being able to
implement.
Was that your experience aswell?
Like you were thrown into allkinds of situations where, like,
I've seen something kind oflike this, but not this exact
thing, in the past yeah, I mean,of course they ask, can you do
this?
Don (09:47):
and we always say yes, of course we can, and then we
figure it out along the way,even you know if we didn't know
how to do it beforehand, but wefigure it out.
Um, so you just have to kind ofbe able to roll with like the
unknown yeah, I love that.
William (10:03):
So I do want to dig really deep into Searchcraft and
just the work that you all aredoing and how things are
pivoting.
But just to kind of startthings off and tee off the
conversation like dramaticchanges right now you know
(10:30):
Google search consumption iskind of on the decline.
I've seen like reports of likesomething like 30% drops in
click-through rates.
You know, with this whole likeAI overview, and you know tools
like all the GPTs, you knowcapturing more query volume, you
know trending upwards.
(10:50):
So from I guess, from yourperspective, is this kind of
where the industry is headed.
Like what is?
What are the new methods, Iguess, for driving traffic to
content, if that makes sense?
Don (11:03):
Sure, yeah, it's definitely .
It is a big problem.
It has wide reachingimplications for content
creators.
Companies, totally.
They're seeing their referraltraffic just like plummet off a
cliff like between the AIsummaries and just them, google
driving referrals to their ownplatforms like YouTube, like the
inbound traffic is reallydiminished, and so a lot of
(11:25):
companies.
Now they're really having toreevaluate their whole
information discovery strategyBefore they could get by like
okay, well, we don't need tomake our search great, we'll
just put this little magnifyingglass up in the corner.
Maybe people use it, maybe theydon't, but it doesn't have to
be great because people willfind it on Google, right, but
(11:53):
Google's not driving traffic toyou anymore, so now they're
having to really take a look atit and be like you know the
search thing.
We haven't invested in it in adecade.
Uh, now we need to really thinkabout like is it is our content
that we're making likediscoverable by people who are
coming with us to us with anintent to find something right.
Like uh, companies like likeYork Times I like to use them,
for example, because they haveexcellent content and their
search is like really bad, likeyou can know the name of an
(12:16):
article or know exactly whatit's about.
Sorry to call them out, but andyou search for it on their site
and you can't find it, like youcan find it on Google.
You can't find it on their site.
Because you can't find it.
Like you can find it on google,you can't find on their site
because they haven't put theinvestment forth to like really
like build that experience up.
Um, and that's kind of wherewe've seen, we've the the
industry going.
(12:36):
We've talked to like a lot ofpeople in specifically in like
content publishing, some of thebigger brands and, um, they're
all kind of having to figurethis out now and it is very much
like a part of their strategyfor like the next 12 to 18
months is like making eitherthrough like keyword search or
AI search or like.
Some of them haven't figuredthat piece out yet, but they all
(12:58):
know they need to do somethingabout it.
William (13:00):
I mean, how long has it been since ChatGPT popped?
I mean, it's not been that long.
And the amount of change.
I mean I was talking to someonethat got laid off.
They worked on a marketing teamand that marketing team just
got like completely just wipedout because a lot of the tool,
like the whole marketingframework that they set up, it's
like almost like no longervalid and you know everything
(13:25):
was.
You know the whole advertising,paid ads, google search, all
the different searches was kindof how they were driving top of
funnel and all that just kind ofalmost changed I don't want to
say overnight, but in the sortof in the context of how
technologists view things, likeit was very quick and there
(13:45):
wasn't a lot of time forpreparation I, I totally agree
with that.
Don (13:50):
It moved much faster than people anticipated.
Um, like, consumers have verymuch gotten used to going to
chat gp key and ask somequestions there now or another.
You know ai answer versus likeclicking through to a site now
that it's like they're gettingpresented their information they
look for right away.
So why would they click throughright?
But that means all of thateffort into SEO optimizations
(14:13):
like out the window.
I think you're going to see alot.
The marketing companies that dostick around.
They're going to have to figureout like okay, well, how do I
optimize now for these LLMs thatare scraping our content?
Like maybe that's, uh, you know, putting things into
structuring content to more oflike a question answer flow,
like that could be a solution,but they've no one's really
(14:33):
figured it out yet yes, so what?
William (14:38):
um, just kind of the genesis of uh search craft.
So what, what were your uhspecific motivations behind
building it?
Like, was it purely, I guess,what I want to get at?
Was it purely technical, likefrustrations and such?
Or did you see like a marketopportunity and just kind of
jump on it Like, okay, this ismissing or it's not as saturated
(15:00):
as it could be.
You know we can do better.
How did that come about?
Don (15:06):
I mean there's a multitude of things I mean.
Over time, at both agencies Iworked at, we had kind of a deep
understanding of how theexisting search platforms work.
You know Solar, elastic, webuilt those implementations out,
algolia but for anything exceptfor the SaaS-hosted ones, if
you want to run it yourself, ittakes a lot of kind-end
(15:31):
expertise to do that.
You have to know aboutclustering and sharding and a
lot of these are written in Javaso you have to learn about okay
well, I've got to be a Java guyand tune my JVM and things like
that.
So we had been used to buildingthings out with that.
But like we're like you knowthere's room for something
(15:52):
better.
Um, those systems, you knowthey, they are very powerful but
they do take a lot of effort.
And since, as I mentioned, alot of our customers were
startups, right, or things wherewe had to move quickly and get
MVPs out and they don't havetime to spend weeks or months or
(16:12):
whatever on searchimplementations.
They have features to ship, sothey don't want to be bogged
down by that.
So we're trying to move fastand take things off the shelf.
But the off-the-shelf thingsthat exist just were not good
enough, right, and for peoplewho aren't trained specifically
(16:33):
in search, there's a learningcurve for them to use these
tools right.
So, like, we're consultants Atthe end of the day, a lot of
times we roll off a project andhave to hand it over, but then
we kept seeing customers comingback and they're like hey, you,
you know, add some synonyms toour index or can you change the
(16:54):
boost weighting on these fieldsor things like that and like
things that they couldn't dothemselves right.
So we're like well, maybethere's an opportunity here
where we can just make thatexperience better better for,
like, developers, better forproduct people, so they don't
have to come and call theconsultants to come tune their
index right.
Um, so we, we wanted to makesomething with a better
(17:19):
experience.
But at the same time, we're like, well, if we're gonna try and
tackle this problem, let'sreimagine it from like the
ground up.
And what that meant is likewell, if I were to write
something new today, where wouldI start?
What language would I startwith?
Like all of those things, wouldI use an off-the-shelf library
(17:39):
to exist?
Probably not, because a lot ofthese products are built on,
actually on the same commonlibrary called.
It's called lucene, I don't knowif you guys are familiar with
that, but it's a search librarythat was developed in 99, I
believe it's had improvementsover the years, but it's still
like the core tech is fairly oldat this time and if you're
(18:04):
building something, you're onlygoing to kind of be as fast and
as performant as that corelibrary.
So we immediately didn't wantto use that.
There's a lot of learnings thatwe got from Lucene, but we
wanted to start with somethingfresh.
So that's why we were like well, let's, you know, rust is
really taking off, let's investin that and see how it goes.
And so we kind of built it onthe side, on like nights and
(18:27):
weekends while we're doing ourconsulting work.
And then about a year ago, uh,it got to the point where, like,
okay, we feel like the core islike feature complete, and if we
wanted to turn this now into anactual, like customer facing
product, we could.
So like, let's, uh, let's talkto some other development
agencies similar to ours and seewhat they would think.
(18:50):
And so we probably had like 50to 100 phone calls with other
dev shops and we're like okay,what are your pain points?
Do you feel the same way thatwe do or are we like totally off
base?
And a lot of people were like,yeah, no, elastic is terrible to
work with and I keep callingthem out, but yeah, they are
they are terrible to work withand I keep calling them out.
But yeah, they are terrible towork with.
It takes too long, we havethose problems and we think if
(19:11):
you were to build somethingbetter, we would totally use it.
William (19:15):
Yeah, that's an interesting decision right there
I got to say if you think ofevery other I can't remember.
Off the top of my head I feellike most search stuff is going
to be C++, but I think elastics,like elastic search it may be
java what is java?
yeah, yeah, it's java.
Um, and just coming in andseeing, okay, everything that
(19:36):
could be a possible competitoris, you know, basically using
c++, but here we have rust.
Yeah it, yeah, rust, yeah, yeahit's, it's bold, it's
adventurous.
I like it.
What like?
Is there any specificadvantages, like out of the gate
, that you notice with Rust,that gives you a distinct
(19:58):
advantage there, like beyond, orjust being like newer and core
and more optimized, I guess?
Don (20:04):
Yeah, I mean we, we totally could have built it like java
or c++.
But, um, you know, going backto like we wanted a new
foundation, um, I, I think youknow building on top of lucene,
like that's what solar andelastic did, and we're like,
okay, let's try something new.
Um, c++, right, you have awhole class of memory issues you
(20:28):
could run into, like nullpointer exceptions.
The burden of memory managementis very much on the developer
to know what they're doing withC++, which there are some great
C++ developers out there.
But that's like you have adanger of that being kind of
like tribalized knowledge, thatmaybe one or two devs in your
team are really strong in thecode base, but like if you
(20:51):
expand the team, maintenancebecomes more difficult.
Right, there's also this likegrowing interest in memory safe
languages.
If you go back to like theBiden administration, they put
up forth a mandate to like shifttowards memory safe languages
to kind of avoid these likewhole class of errors that
people are getting into.
And Rust just gives you thatsafety, both thread and memory
(21:14):
safety, out of the box.
The compiler for Rust is likevery nice, it's really helpful
with you, know, when you're inthe code and writing and like
providing you kind of guidanceof where to go.
I found that to be a bigdifferentiator.
Um, their whole like system ofpattern matching and like
algebraic types, and those arereally pleasurable to work with
(21:34):
and their inheritance model withwith traits, is really really
well thought out, like it's verywell kind of designed and
thought out language yeah, it'sinteresting too, because I think
, I think this has happened butlenis decided to, you know,
incorporate rust in the linuxkernel, I'm pretty sure.
William (21:54):
Well, why not c++?
Uh, he, everything, I don'tknow.
I just think that that'sinteresting because of course he
he's why we have linuxessentially, and like why Git
exists.
So he, you know, probably has agood opinion on, you know, the
solid things to incorporate.
So yeah, you kind of got mewanting to, you know, take a
(22:17):
peek at Rust and try, you know,do some learnings there.
Don (22:21):
Yeah, it's really, it's like they don't overload the
standard library.
So there's this whole packagesystem with cargo and their,
their crates, which are reallynice.
Um, I would say, you know, Idid a lot of like development in
other languages that havepackage vendors and I would say
rust is probably the mostconvenient to work with as far
as, like you know, dependencyresolution and things like that.
(22:42):
Um, so yeah, I would say,definitely give it a try.
William (22:46):
Um and I guess so.
Not to, I guess, linger onprogramming languages, but, um,
yeah, we shouldn't definitely.
But uh, searchcraft has, youknow, there's this interesting
dual nature kind of like youalluded to.
So you've you, you're both anagency, you know, providing, you
know services, if you will, andyou're also building a, a
(23:10):
product that's, you know,positioned as this, like drop-in
replacement for elastic search.
Uh, how so does I guess?
Uh, one benefit that just popsout, you know, on the top of my
head, is like, oh, the agencywork and the learnings from the
agency work, or like, kind oflike a direct um feedback loop,
(23:32):
or you know a way to like informyour product development for
the replacement that you'rebuilding.
Is that, would that be true?
Don (23:40):
oh, yeah, very much so, um, like we.
So my co-founder and I, we bothcome from that services
background, um, and so you know,we've been the people that you
know non-technical founderswould lean on.
They're like, okay, well, Idon't really have someone who
can stand up this application,can you guys help us?
Um, and so we wanted to carrythat over, that mentality and
(24:03):
that spirit when we came, whenwe founded searchcraft, like and
a lot of our competition.
They don't do that.
They're just like we're a sascompany.
You either download it or youbuy it and then good luck, right
, like you might be able to hirelike a third party, like
consultant, to help help you,right, maybe someone who's
certified and like elastic orwhatever.
Um, but we're very we're muchmore interested in having close,
(24:26):
close relationships with ourcustomers, um, so we can know,
like, what are the problems youguys are specifically trying to
solve and like are there painpoints that, like, we can take
from and learn and like maybeinform our product robot
decisions based on like problemsthey're coming across or maybe
features that we're missing,things like that.
(24:47):
So, yeah, all of our customerswere like, when they kind of
sign up and they get going andthey start building.
We're like, yeah, haveconversations with us, you know,
talk to us on our Discord andif you get stuck, like we can
help you.
William (25:01):
Gotcha, how do you balance?
I mean, I know that being astartup and just being like
young and lean, you know there'sjust only so many bodies to go
around, but how do you balanceresources between like services
and the product, or is that?
Is it to that part?
Don (25:17):
Yeah, and we're a small team, so I'm not so young
anymore, but I do.
I do work a lot of hours.
I have a lot of coffee.
Yeah, no, I no.
I mean eventually, you know, asas we grow, we plan on kind of
scaling the team up, but, um,that the services part will
(25:38):
always remain important to kindof like the core of who we are.
Um, and just like making surethat people don't get stuck.
William (25:48):
Gotcha and I did see.
So you actually went andchecked out the repo but you
launched a container image and Ithink I saw something to the
effect of it's like 90% or sosmaller than elastic search I
(26:09):
container image.
I just thought that was like,yeah, it's a lot.
Yeah, 90 percent's a solidnumber you can do the math.
Don (26:15):
You can go, look at elastic search docker container and
look at ours and you'll see.
And most of ours is actuallydebian.
William (26:21):
It's not the actual app that's taking up the space
that's so funny because Iactually just dockerized an
application recently and, um,yeah, I use debbie and slim, the
most slim down version.
And then you know build time.
You, you know install your allthe packages you have, like your
build time stuff you install,that's consistent, and you have
(26:41):
some runtime things you do aswell, like your entry point and,
yeah, the the hardest or thelongest part about it is
building that base image up andonly getting installed on it
what you need.
You know, that's where all thetotally space and time comes
from.
Um, yeah, but that's awesome.
Is that?
Is that open?
I didn't look at the licensingor anything.
How does that does that takecommunity contributions or like?
(27:03):
What is the stance on that?
Don (27:05):
it doesn't we.
So we, our core is not opensource.
All of our integrations aregoing to be but rsdks and and
our clients and all that, butthe core is not, and part of the
reason for that is, like opensource is very important to us.
We didn't want to make adecision to open something up
and then possibly pull that backlater.
Um, and we had some concernswith, you know, some of the
larger cloud providers and kindof what make a decision to open
(27:27):
something up and then possiblypull that back later.
And we had some concerns withsome of the larger cloud
providers and kind of whathappened with Elastic with their
license.
So we're like, well, let's notopen that door at this time, but
we still want people to be ableto use it and not have to pay.
So that's why we have it.
It's free to use.
You can download it.
We also have a free cloudhosted version, which we call
(27:51):
SearchRef Cloud.
That's got a free tier as well.
You have options as a developer.
You can either run it yourself,and that's totally fine, or you
can use our hosted option andhave a free tier.
That's fine too.
We really don't uh care likewhich way you go, as long as you
like, try it out.
Um, another reason why wedidn't open it up is, like it's
(28:13):
a very search, is a very likeniche problem, and we were going
to probably be the solecontributors to the repo anyway,
so it just really didn't makesense for us at this time.
Eyvonne (28:26):
So, from a product side , as you're thinking about
Searchcraft like what's your,what's your ideal use case, or
if somebody's thinking like isthis a good fit for me, like
what's what's kind of your, youknow the ideal either problem
you're solving for or situationin which Searchcraft's a perfect
fit.
Don (28:46):
Oh sure, yeah.
So unlike Elastic, we're nottrying to solve everyone's
problems.
So Elastic is really good at alot of different things, so like
log search, content search, etcetera, we're not trying to do
like log search specifically.
So if you're looking for a logsearch, you can hit me up.
I can make some recommendations, but don't use search craft for
that.
We are very focused on contentsearch, so human-oriented
(29:11):
natural language content, andthe reason I say that is we
optimize a lot of our featuresaround that and making sure that
you can turn on these featuresat query time and still have
very good performance for thingslike typo tolerance or fuzzy
matching and whatnot, whereassome of our competitors, you
know, if you want to turn on thefull suite of features, you
(29:33):
kind of have a trade-off.
We're like, okay, I'm going tohave a performance issue when
I'm getting in the millions ofdocuments.
Eyvonne (29:39):
Now, right, so very much the content providers,
media, long form content, thosethose kinds of arenas yeah, like
knowledge bases, documentationis great.
Don (29:51):
Uh, you know publishing content, like you know, news
organizations um, e-commerce islike.
This is kind of one of thethings.
William (30:13):
I was really super excited.
I mean a lot of it.
You know, a lot of the stuffwe've been talking about is
really relevant and exciting,but, um, just kind of the
current state of the market, um,like fundraising, for instance,
like you know, let's I guesslet's just talk about it every,
every founder is dealing with itright now fundraising is hard.
Um, the data is, you know,pretty stark.
(30:36):
Like, basically, unless itseems like, unless you have ai
prominently in your pitch, youhave ai coming out of your palms
and everything's AI it issignificantly harder to raise
capital in 2025.
And we're seeing anybody thatlooks at the news for more than
(30:58):
five minutes is seeing these AIcompanies raise money and it
just massive valuations with youknow where the math just
doesn't even make sense, youknow, while non, non AI
companies tend to struggle.
Do you have any experience hereand, if so, like?
What has been your experiencewith fundraising in this
(31:19):
environment, or what are yourthoughts?
Don (31:46):
no-transcript.
Understand that ai does notalways need to be the hammer
that drives on the elf, right,um, but there's a good number of
them that were really feelingthe pressure to make sure their
investments like have an aiangle to them, like from their
partners or whatever right.
So it's a question that was notasked so much at the beginning
(32:08):
of when we started our fundraisebut, like now and later on in
the process, it very much was um, so like our core product, um,
when we built it, did not relyon ai at all, uh, but the
landscape, as you say, hasshifted since we built that core
.
So, um, I think, um, you knowit's ai is a very transformative
(32:31):
technology.
It's fundamentally changing thelandscape of business, um, and
that has impacts on andeveryone's businesses.
Um, our view on ai is that, like, we're applying it where it
makes sense to kind of augmentand improve the overall
experience.
So, like AI is very good atapplying transformations or
analysis over an existing dataset, so we're utilizing it to
(32:55):
power, like some of those typesof features like trend analysis
and like generative dataenrichment.
We're also doing naturallanguage processing, semantic
search via our SDKs.
So that's going to be a layerwhere, if customers want that
kind of like transformative,like summary response, they can
(33:18):
use that and we'll have thatfeature available to them.
But if they don't want it, theydon't have to have it, because
some of our customers actuallylike, don't want ai like they've
.
They've said like we just want asmaller vaster elastic, we
don't want an ai feature and solike that's kind of why we don't
ship that as part of our core,but we do have it available in
both, like our cloud dashboardand our sdk, so like people can
(33:40):
pull it in and use it or theycan like opt out of it and not
um, but we're kind of a bigbeliever of choice.
So that's kind of like we'renot going to force that on
customers if they don't want it,but it is there if they do
gotcha.
William (33:54):
Yeah, that makes a lot of sense.
I really I could be justtotally wrong here.
But just an observation.
It seems like you have likereal the, you know the
foundational model companies,like companies that are actually
building ai, or they build, dothey procure and produce the
infrastructure that powers it,and then you have, like it's
(34:14):
almost like everybody else iskind of like using it or
incorporate, eitherincorporating it with like
existing features or making itas they go in and they pitch, or
they try to get funding, like,yeah, we have this awesome thing
, but we're using AI to enhanceit like X, y and Z, because,
let's face it, if you'rebuilding AI, there is a
(34:37):
difference between those twothings, I think.
Don (34:39):
Oh for sure, yeah, and these businesses are kind of
just like they're going to wrapsome llm or whatever, like do
what differentiates them from,like the other people who are
doing the same thing, right?
William (34:56):
um, I think a lot of those are going to fall away
after a while, after the hypekind of settles a bit yeah, and
I think that your, your responsethere is, I thought, pretty
thoughtful because it's like oneof you know, I just the media,
youtube everywhere is sosaturated with AI like, hey, you
can vibe code this startup andthat's just not the case.
(35:18):
Maybe one day it will be andvibe coding is a real thing, but
it's got its limits, of course.
But it's got its limits, ofcourse.
You know you still have to beable to validate and know and
like, look at what you're doingand you still need expertise.
Yvonne and I were actuallytalking about that this morning
um on, kind of like, the stateof that um in the market.
But you know, positioning likesearch craft in relation to the
(35:41):
AI wave, I think the kind of theway you pitched it was wasn't
disingenuous, it was hey, thisis, this is what it is and this
is how it's getting used.
Don (35:50):
So pretty, yeah, I mean pretty awesome as a tech company
, you do have to be able toanswer that question, because
people will ask you like okay,well, what about AI?
Why should I use your productif you don't have AI?
William (36:01):
so to check a box it's like the new, the new security
team.
Yeah, but yeah also like justbecause you produce an mcp
server doesn't mean that you'rean ai company, you know yes, I
mean we're not in the modelbusiness.
Don (36:19):
We don't want to be and like we actually don't prescribe
a specific model for, like, ourimplementation.
Uh, we let people can.
They can bring their own ifthey want.
I mean, we can like stand thefull thing up for them, but if
they want to bring their ownbecause they're already paying
for like mistral or clot orwhatever they can.
So so I think we still don'tknow yet who's gonna be like the
(36:42):
winners and losers on the modelside, right?
William (36:47):
yeah, definitely, um.
So I guess, like, as we nearthe end of the um the hour, um.
So what?
Just out of out of curiosity,what's your favorite developer
tool outside of search, or whatdo you like working with?
Don (37:04):
um, oh wow, let me, let me think here uh I was, I.
William (37:10):
I was actually asked this question the other day and
I froze.
Don (37:13):
I was like I don't know um, does it have to be okay?
Can it be like a, a thing youpay for, like a sass thing?
Is that okay?
Oh yeah, I'm a big fan ofPlausible.
I don't know if you guys haveused Plausible at all.
William (37:31):
Uh-uh.
Don (37:32):
So they're an analytics platform, but their whole thing
is not tracking people and verymuch anonymous analytic
reporting.
So we pretty much use them onall of our marketing sites that
we build, because it's just kindof less shady than the Google
(37:52):
Analytics and it also means youcan get by not having to do
those giant consent banners thatyou have to do for GDPR, which
is really nice, because I hatemaking those.
William (38:04):
Yeah, okay, next one.
What's your biggest technicalchallenge, looking back, that
you've overcome in building thenew uh product?
Don (38:18):
okay, um, for me it's like having to do a lot of the.
I mean, I I'm always buildingthings I haven't had to build
before, so at the beginning itwas like learning about how do I
build this core search library.
But once we got past thatbuilding up the whole tooling
(38:43):
and infrastructure around makinga software sas company um, we
have like internal, likeanalytics tracking for like what
people are searching on andthings like that, and they need
to be scalable in a way where wecould, um search across or like
track analytics across, likemultiple search index clusters,
(39:04):
but have it in a, a centraldatabase.
So we went with a productcalled ClickHouse.
I don't know if you guys arefamiliar with ClickHouse, but
the documentation is not great.
So, figuring out how to standthat up and scale that on our
AWS cloud, it took a while, andthen we'd put it up get it up in
(39:25):
production and we'd startgetting traffic into it.
And then we put it up, get up inproduction and we'd start
getting traffic into it and then, like we would have issues.
I'm like, and this thing, likeit, gets used by companies like
ebay and netflix and these hugecompanies.
So we know it's, it can scale.
We just don't know how to liketune it properly.
So there was quite a bit of alearning curve with that it
(39:50):
properly.
William (39:50):
So there was quite a bit of a learning curve with
that gotcha and I guess my lastlightning round question is if
you can give you know aspiringdevelopers that are just kind of
getting their you know, dippingtheir toes in the search um
arena, you know what would beone piece of advice, uh, that
you would give to them aboutchoosing search infrastructure.
Don (40:11):
Well, hopefully you don't have to become an expert in
search, because if you're havingto become an expert in search,
then I didn't do my job well.
So we want it to be anexperience where people can just
pull it off the shelf, get itup and running and be on the way
and build things that actuallymatter to their product, like
building out their productfeatures, not building search.
(40:31):
So it really should not besomething that people should
have to deal with like, um, justkind of like like
authentication, like you don'twant to be dealing with those
types of problems.
You need to deal with problemsthat make your product unique
and let, kind of like, theexperts deal with those other
issues.
Right, that's my advice.
(40:52):
Anyway, like I wouldn't if Iwas coming in, I now, like I
wouldn't want to learn all theyhave to learn to build search,
right, like, just do somethingoff the shelf.
William (41:04):
I can honestly say search and authentication,
identity and access managementare two areas where I just want
to off the shelf.
I can honestly say search andauthentication, identity and
access management are two areaswhere I just want to run and
hide.
Don (41:13):
Yeah, I've built out off systems and most people should
not do that.
They should absolutely not dothat work.
William (41:21):
Totally agree.
You got anything else, yvonne.
Eyvonne (41:24):
Before we wrap up no, I'm all good Love to know where
to find Dawn online and ifpeople want to know more where
they can look.
William (41:32):
Yeah, where can we find you?
Don (41:37):
So you know I'm on LinkedIn and Blue Sky primarily.
I'm not really on Twitteranymore, but if you go to
dawnmckinnondev it has links toboth Searchcraft and my LinkedIn
and my GitHub and my Boost.
William (41:51):
Awesome and I'll link that.
Yeah, I'll definitely link allthose, especially the GitHub,
and, yeah, everybody out therecheck it out and thank you so
much for coming on and weappreciate the time.
has wide-reaching implications for content
creators.
Companies they're seeing theirreferral traffic just like
plummet off a cliff like betweenthe AI summaries and just them
Google driving referrals totheir own platforms like YouTube
.
Like the inbound traffic isreally diminished and so a lot
of companies now like they'rereally having to reevaluate like
their whole like informationdiscovery strategy.
William (00:34):
Welcome back to the Cloud Gambit, where we unravel
the state of cloud computingmarkets strategy, emerging
trends or whatever else.
To the list, I'm William andwith me, as always, is Yvonne.
Eyvonne (00:54):
How are you doing today , yvonne?
How's everything going inKentucky?
Good, the sun's shining heretoday.
It's warm outside, which is anice change.
It's taken until June for it toget warm and sunny, but it's a
warm, sunny day, and so that'sgood.
William (01:06):
Yeah, and you know what's great about it is it's
not humid.
So usually around this time ofyear, as soon as you hit June,
the humidity about kills you,and right now the humidity's low
.
It would be a great day to takea walk.
Eyvonne (01:21):
Yep, that is a bike ride is on my list as soon as we
wrap this thing up thisafternoon.
William (01:27):
Awesome.
So today we're diving into oneof the most, I would say,
fundamental aspects really oflike the modern web search, but
not just any search discussion.
We're talking about how theentire landscape is kind of
shifting beneath our feet andwhat that means for developers,
(01:48):
businesses alike, and our guesttoday is Don McKinnon,
co-founder and CTO ofSearchcraft.
How are you doing today, Don?
Don (01:57):
I'm doing great, great to meet you all.
I have to say, the reason Idon't live in the South is
because of the humidity, sothat's why I'm in Colorado.
William (02:08):
It's very smart.
Yeah, you have the dry sort ofthe dry climate.
You know the carry around lipbalm in your pocket climate, but
we've got the scorching.
You know if you walk outsideyou're sweating in a minute
climate.
So I would gladly tradeclimates for sure.
So I guess, starting out, don,let's start out with your story.
(02:30):
Take us through your journey.
Who you are and how did you endup co-founding a search company
?
What were the, I guess, thepivotal moments that led you
here?
Don (02:44):
Sure, let's see how far back do I go?
I would say you know, I callmyself a builder.
That's kind of like mybackground and strength.
I've been working in techprofessionally since 98.
My first job was at a startupduring the dot-com era.
Back then it was Pearl and PHPand and tickle was what I was
(03:07):
writing, um.
So.
But I kind of got my start onthe tech side, um, working in
media and publishing, um.
So back then in like the early2000s, late 90s, media and
publishing was like really bigum, those companies were huge
and they had like thousands ofemployees in like manhattan and
elsewhere like huge offices,right.
(03:28):
So I I did about 12 years at acompany called time inc and they
owned a bunch of brands peopleare familiar with, even if
they're not familiar with thatcompany called like time and
sports, illust, fortune, people,those brands, right.
So I actually started workingfor them from a subsidiary that
(03:49):
they had acquired in Boulder.
There was like a winter sportssubsidiary.
They bought them and at thetime I was one of the few people
in Colorado that had a lot ofexperience with Tickle, which is
a language you don't hear muchabout these days, but back then
the platform was called StoryServer, vignette Story Server,
(04:10):
and it ran much of the web backthen.
So they brought me in to belike okay, well, we acquired
this company, they're onColdFusion, we're going to
migrate from ColdFusion to StoryServer and we need someone
who's good at Tickle.
So they brought me in.
We need someone who's going totickle.
So they brought me in and Ikind of ran the dev team for
those sites.
(04:30):
So I did that for a bit andthen later you know publishing
industry they like to kind ofconsolidate and, you know, sell
each other their brands.
So they sold off the ones thatwere here in Boulder.
So I moved to New York and Iworked there for a bit on some
other brands.
I worked there for a bit onsome other brands and so that's
kind of when my role within theorganization shifted from just
being a developer to being likean engineering manager.
(04:51):
So I started running dev teamsand getting more involved in
like stakeholder conversations,working with product managers
and that whole thing.
And I worked for that companyfor about 12 years, um, but then
every pretty much every year Iworked there, they were doing
(05:11):
layoffs, um, and then one yearthey were like, okay, well,
we're gonna just cut your wholedepartment, so.
But it ended up being kind oflike a political situation in
that, um, somebody else at thedirector level that was the same
level as my boss wanted ourgroup, but they didn't want to
keep my boss, so they laideveryone off and then re-offered
us all our jobs, except for himthe next day to come back, and
(05:35):
so we're like well, you know,we're kind of loyal to our boss.
So we all said no and went off.
Um, so that's when I left thereand but that was kind of a
motivator for me to not work atthese big companies anymore.
Like you really have no controlover your kind of your destiny.
You know, if it doesn't matterhow hard you work, if you're
just like a number on a budget,like you could be cut right.
(05:58):
So I got into consulting.
At that time that was like 2013.
So first there's like justfreelance consulting back to
publishing industry people.
So, but not only back at time,but also condi, nast and other
publishers in new york.
Um, so I at that time I I wentoff on my own for a little bit
(06:21):
and then I joined a consultingagency that a guy I knew started
.
He was a PM that I had workedwith previously.
He's like hey, you know, we havethese big projects coming up
and we really need someone whokind of knows everything about
like you know how to stand up atech stack and architect and all
that.
So he brought me in and I ranthe engineering team for that
(06:43):
consulting company and weconsulted for companies like
Sony Music and Condé and Reutersand places like that, building
out CMSs, doing like datamigrations and then later
building out like a bunch oflike Elasticsearch
implementations, and so that'skind of where I got into
(07:15):
learning about search and likeit takes to deal with relevancy
issues and data indexing andcluster management things like
that.
So I did that agency for a bit.
And then I ended up joininganother agency with my current
co-founder of Searchcraft.
His name's Sharpie, so he'smore of a product and design
oriented person and I'm kind oflike the engineering guy, and
then we can kind of just combineforces and we did the
(07:35):
consulting thing for a bit.
But then on the side we werekind of building Searchcraft and
about a year ago we decided,okay, maybe we should wind down
the agency and try and do thisproduct thing full-time.
William (07:52):
Sounds like a journey, quite the journey.
That consulting work, grindingin the consulting space, is a
really good experience becauseyou come in usually as a
consultant, you come into whatsome might refer to as a hot
mess or just a reallycomplicated problem, and the
point being that that companymay not have the engineering
(08:12):
staff or the wherewithal tosolve or migrate or do something
.
You know that requires verydeep expertise.
So when you come into thosetypes of environments you don't,
and then everything's like oh,we need it done as fast as
possible when you you know wegotta meet those deadlines.
Don (08:28):
so yeah, that was kind of our niche, like we got known for
that um at our agency is, isthat, like a lot of
non-technical founders wouldbring us in because they have a
product vision but they didn'tknow what it meant to like scale
from zero and build an mvp andthey wanted somebody who had a
reputation for moving quicklyand that could handle everything
.
Eyvonne (08:48):
And even at some startups that we worked at where
they had technical people butthey couldn't move as fast as we
could, right, so they wouldcall us One of the things a lot
of folks don't understand who'venever been in consulting or
worked in and around it isreally the trick in consulting
work is being able to learn andthink on your feet.
(09:11):
Oftentimes, when you work for aconsultant, you may have a core
of experience, but very oftenyou get thrown into a situation
where you don't have the exactspecific expertise that you need
to solve the problem.
It's more about being able tounderstand what's going on,
pivot and then learn what youneed to do.
(09:33):
Learn what you need to knowquickly and being able to
implement.
Was that your experience aswell?
Like you were thrown into allkinds of situations where, like,
I've seen something kind oflike this, but not this exact
thing, in the past yeah, I mean,of course they ask, can you do
this?
Don (09:47):
and we always say yes, of course we can, and then we
figure it out along the way,even you know if we didn't know
how to do it beforehand, but wefigure it out.
Um, so you just have to kind ofbe able to roll with like the
unknown yeah, I love that.
William (10:03):
So I do want to dig really deep into Searchcraft and
just the work that you all aredoing and how things are
pivoting.
But just to kind of startthings off and tee off the
conversation like dramaticchanges right now you know
(10:30):
Google search consumption iskind of on the decline.
I've seen like reports of likesomething like 30% drops in
click-through rates.
You know, with this whole likeAI overview, and you know tools
like all the GPTs, you knowcapturing more query volume, you
know trending upwards.
(10:50):
So from I guess, from yourperspective, is this kind of
where the industry is headed.
Like what is?
What are the new methods, Iguess, for driving traffic to
content, if that makes sense?
Don (11:03):
Sure, yeah, it's definitely .
It is a big problem.
It has wide reachingimplications for content
creators.
Companies, totally.
They're seeing their referraltraffic just like plummet off a
cliff like between the AIsummaries and just them, google
driving referrals to their ownplatforms like YouTube, like the
inbound traffic is reallydiminished, and so a lot of
(11:25):
companies.
Now they're really having toreevaluate their whole
information discovery strategyBefore they could get by like
okay, well, we don't need tomake our search great, we'll
just put this little magnifyingglass up in the corner.
Maybe people use it, maybe theydon't, but it doesn't have to
be great because people willfind it on Google, right, but
(11:53):
Google's not driving traffic toyou anymore, so now they're
having to really take a look atit and be like you know the
search thing.
We haven't invested in it in adecade.
Uh, now we need to really thinkabout like is it is our content
that we're making likediscoverable by people who are
coming with us to us with anintent to find something right.
Like uh, companies like likeYork Times I like to use them,
for example, because they haveexcellent content and their
search is like really bad, likeyou can know the name of an
(12:16):
article or know exactly whatit's about.
Sorry to call them out, but andyou search for it on their site
and you can't find it, like youcan find it on Google.
You can't find it on their site.
Because you can't find it.
Like you can find it on google,you can't find on their site
because they haven't put theinvestment forth to like really
like build that experience up.
Um, and that's kind of wherewe've seen, we've the the
industry going.
(12:36):
We've talked to like a lot ofpeople in specifically in like
content publishing, some of thebigger brands and, um, they're
all kind of having to figurethis out now and it is very much
like a part of their strategyfor like the next 12 to 18
months is like making eitherthrough like keyword search or
AI search or like.
Some of them haven't figuredthat piece out yet, but they all
(12:58):
know they need to do somethingabout it.
William (13:00):
I mean, how long has it been since ChatGPT popped?
I mean, it's not been that long.
And the amount of change.
I mean I was talking to someonethat got laid off.
They worked on a marketing teamand that marketing team just
got like completely just wipedout because a lot of the tool,
like the whole marketingframework that they set up, it's
like almost like no longervalid and you know everything
(13:25):
was.
You know the whole advertising,paid ads, google search, all
the different searches was kindof how they were driving top of
funnel and all that just kind ofalmost changed I don't want to
say overnight, but in the sortof in the context of how
technologists view things, likeit was very quick and there
(13:45):
wasn't a lot of time forpreparation I, I totally agree
with that.
Don (13:50):
It moved much faster than people anticipated.
Um, like, consumers have verymuch gotten used to going to
chat gp key and ask somequestions there now or another.
You know ai answer versus likeclicking through to a site now
that it's like they're gettingpresented their information they
look for right away.
So why would they click throughright?
But that means all of thateffort into SEO optimizations
(14:13):
like out the window.
I think you're going to see alot.
The marketing companies that dostick around.
They're going to have to figureout like okay, well, how do I
optimize now for these LLMs thatare scraping our content?
Like maybe that's, uh, you know, putting things into
structuring content to more oflike a question answer flow,
like that could be a solution,but they've no one's really
(14:33):
figured it out yet yes, so what?
William (14:38):
um, just kind of the genesis of uh search craft.
So what, what were your uhspecific motivations behind
building it?
Like, was it purely, I guess,what I want to get at?
Was it purely technical, likefrustrations and such?
Or did you see like a marketopportunity and just kind of
jump on it Like, okay, this ismissing or it's not as saturated
(15:00):
as it could be.
You know we can do better.
How did that come about?
Don (15:06):
I mean there's a multitude of things I mean.
Over time, at both agencies Iworked at, we had kind of a deep
understanding of how theexisting search platforms work.
You know Solar, elastic, webuilt those implementations out,
algolia but for anything exceptfor the SaaS-hosted ones, if
you want to run it yourself, ittakes a lot of kind-end
(15:31):
expertise to do that.
You have to know aboutclustering and sharding and a
lot of these are written in Javaso you have to learn about okay
well, I've got to be a Java guyand tune my JVM and things like
that.
So we had been used to buildingthings out with that.
But like we're like you knowthere's room for something
(15:52):
better.
Um, those systems, you knowthey, they are very powerful but
they do take a lot of effort.
And since, as I mentioned, alot of our customers were
startups, right, or things wherewe had to move quickly and get
MVPs out and they don't havetime to spend weeks or months or
(16:12):
whatever on searchimplementations.
They have features to ship, sothey don't want to be bogged
down by that.
So we're trying to move fastand take things off the shelf.
But the off-the-shelf thingsthat exist just were not good
enough, right, and for peoplewho aren't trained specifically
(16:33):
in search, there's a learningcurve for them to use these
tools right.
So, like, we're consultants Atthe end of the day, a lot of
times we roll off a project andhave to hand it over, but then
we kept seeing customers comingback and they're like hey, you,
you know, add some synonyms toour index or can you change the
(16:54):
boost weighting on these fieldsor things like that and like
things that they couldn't dothemselves right.
So we're like well, maybethere's an opportunity here
where we can just make thatexperience better better for,
like, developers, better forproduct people, so they don't
have to come and call theconsultants to come tune their
index right.
Um, so we, we wanted to makesomething with a better
(17:19):
experience.
But at the same time, we're like, well, if we're gonna try and
tackle this problem, let'sreimagine it from like the
ground up.
And what that meant is likewell, if I were to write
something new today, where wouldI start?
What language would I startwith?
Like all of those things, wouldI use an off-the-shelf library
(17:39):
to exist?
Probably not, because a lot ofthese products are built on,
actually on the same commonlibrary called.
It's called lucene, I don't knowif you guys are familiar with
that, but it's a search librarythat was developed in 99, I
believe it's had improvementsover the years, but it's still
like the core tech is fairly oldat this time and if you're
(18:04):
building something, you're onlygoing to kind of be as fast and
as performant as that corelibrary.
So we immediately didn't wantto use that.
There's a lot of learnings thatwe got from Lucene, but we
wanted to start with somethingfresh.
So that's why we were like well, let's, you know, rust is
really taking off, let's investin that and see how it goes.
And so we kind of built it onthe side, on like nights and
(18:27):
weekends while we're doing ourconsulting work.
And then about a year ago, uh,it got to the point where, like,
okay, we feel like the core islike feature complete, and if we
wanted to turn this now into anactual, like customer facing
product, we could.
So like, let's, uh, let's talkto some other development
agencies similar to ours and seewhat they would think.
(18:50):
And so we probably had like 50to 100 phone calls with other
dev shops and we're like okay,what are your pain points?
Do you feel the same way thatwe do or are we like totally off
base?
And a lot of people were like,yeah, no, elastic is terrible to
work with and I keep callingthem out, but yeah, they are
they are terrible to work withand I keep calling them out.
But yeah, they are terrible towork with.
It takes too long, we havethose problems and we think if
(19:11):
you were to build somethingbetter, we would totally use it.
William (19:15):
Yeah, that's an interesting decision right there
I got to say if you think ofevery other I can't remember.
Off the top of my head I feellike most search stuff is going
to be C++, but I think elastics,like elastic search it may be
java what is java?
yeah, yeah, it's java.
Um, and just coming in andseeing, okay, everything that
(19:36):
could be a possible competitoris, you know, basically using
c++, but here we have rust.
Yeah it, yeah, rust, yeah, yeahit's, it's bold, it's
adventurous.
I like it.
What like?
Is there any specificadvantages, like out of the gate
, that you notice with Rust,that gives you a distinct
(19:58):
advantage there, like beyond, orjust being like newer and core
and more optimized, I guess?
Don (20:04):
Yeah, I mean we, we totally could have built it like java
or c++.
But, um, you know, going backto like we wanted a new
foundation, um, I, I think youknow building on top of lucene,
like that's what solar andelastic did, and we're like,
okay, let's try something new.
Um, c++, right, you have awhole class of memory issues you
(20:28):
could run into, like nullpointer exceptions.
The burden of memory managementis very much on the developer
to know what they're doing withC++, which there are some great
C++ developers out there.
But that's like you have adanger of that being kind of
like tribalized knowledge, thatmaybe one or two devs in your
team are really strong in thecode base, but like if you
(20:51):
expand the team, maintenancebecomes more difficult.
Right, there's also this likegrowing interest in memory safe
languages.
If you go back to like theBiden administration, they put
up forth a mandate to like shifttowards memory safe languages
to kind of avoid these likewhole class of errors that
people are getting into.
And Rust just gives you thatsafety, both thread and memory
(21:14):
safety, out of the box.
The compiler for Rust is likevery nice, it's really helpful
with you, know, when you're inthe code and writing and like
providing you kind of guidanceof where to go.
I found that to be a bigdifferentiator.
Um, their whole like system ofpattern matching and like
algebraic types, and those arereally pleasurable to work with
(21:34):
and their inheritance model withwith traits, is really really
well thought out, like it's verywell kind of designed and
thought out language yeah, it'sinteresting too, because I think
, I think this has happened butlenis decided to, you know,
incorporate rust in the linuxkernel, I'm pretty sure.
William (21:54):
Well, why not c++?
Uh, he, everything, I don'tknow.
I just think that that'sinteresting because of course he
he's why we have linuxessentially, and like why Git
exists.
So he, you know, probably has agood opinion on, you know, the
solid things to incorporate.
So yeah, you kind of got mewanting to, you know, take a
(22:17):
peek at Rust and try, you know,do some learnings there.
Don (22:21):
Yeah, it's really, it's like they don't overload the
standard library.
So there's this whole packagesystem with cargo and their,
their crates, which are reallynice.
Um, I would say, you know, Idid a lot of like development in
other languages that havepackage vendors and I would say
rust is probably the mostconvenient to work with as far
as, like you know, dependencyresolution and things like that.
(22:42):
Um, so yeah, I would say,definitely give it a try.
William (22:46):
Um and I guess so.
Not to, I guess, linger onprogramming languages, but, um,
yeah, we shouldn't definitely.
But uh, searchcraft has, youknow, there's this interesting
dual nature kind of like youalluded to.
So you've you, you're both anagency, you know, providing, you
know services, if you will, andyou're also building a, a
(23:10):
product that's, you know,positioned as this, like drop-in
replacement for elastic search.
Uh, how so does I guess?
Uh, one benefit that just popsout, you know, on the top of my
head, is like, oh, the agencywork and the learnings from the
agency work, or like, kind oflike a direct um feedback loop,
(23:32):
or you know a way to like informyour product development for
the replacement that you'rebuilding.
Is that, would that be true?
Don (23:40):
oh, yeah, very much so, um, like we.
So my co-founder and I, we bothcome from that services
background, um, and so you know,we've been the people that you
know non-technical founderswould lean on.
They're like, okay, well, Idon't really have someone who
can stand up this application,can you guys help us?
Um, and so we wanted to carrythat over, that mentality and
(24:03):
that spirit when we came, whenwe founded searchcraft, like and
a lot of our competition.
They don't do that.
They're just like we're a sascompany.
You either download it or youbuy it and then good luck, right
, like you might be able to hirelike a third party, like
consultant, to help help you,right, maybe someone who's
certified and like elastic orwhatever.
Um, but we're very we're muchmore interested in having close,
(24:26):
close relationships with ourcustomers, um, so we can know,
like, what are the problems youguys are specifically trying to
solve and like are there painpoints that, like, we can take
from and learn and like maybeinform our product robot
decisions based on like problemsthey're coming across or maybe
features that we're missing,things like that.
(24:47):
So, yeah, all of our customerswere like, when they kind of
sign up and they get going andthey start building.
We're like, yeah, haveconversations with us, you know,
talk to us on our Discord andif you get stuck, like we can
help you.
William (25:01):
Gotcha, how do you balance?
I mean, I know that being astartup and just being like
young and lean, you know there'sjust only so many bodies to go
around, but how do you balanceresources between like services
and the product, or is that?
Is it to that part?
Don (25:17):
Yeah, and we're a small team, so I'm not so young
anymore, but I do.
I do work a lot of hours.
I have a lot of coffee.
Yeah, no, I no.
I mean eventually, you know, asas we grow, we plan on kind of
scaling the team up, but, um,that the services part will
(25:38):
always remain important to kindof like the core of who we are.
Um, and just like making surethat people don't get stuck.
William (25:48):
Gotcha and I did see.
So you actually went andchecked out the repo but you
launched a container image and Ithink I saw something to the
effect of it's like 90% or sosmaller than elastic search I
(26:09):
container image.
I just thought that was like,yeah, it's a lot.
Yeah, 90 percent's a solidnumber you can do the math.
Don (26:15):
You can go, look at elastic search docker container and
look at ours and you'll see.
And most of ours is actuallydebian.
William (26:21):
It's not the actual app that's taking up the space
that's so funny because Iactually just dockerized an
application recently and, um,yeah, I use debbie and slim, the
most slim down version.
And then you know build time.
You, you know install your allthe packages you have, like your
build time stuff you install,that's consistent, and you have
(26:41):
some runtime things you do aswell, like your entry point and,
yeah, the the hardest or thelongest part about it is
building that base image up andonly getting installed on it
what you need.
You know, that's where all thetotally space and time comes
from.
Um, yeah, but that's awesome.
Is that?
Is that open?
I didn't look at the licensingor anything.
How does that does that takecommunity contributions or like?
(27:03):
What is the stance on that?
Don (27:05):
it doesn't we.
So we, our core is not opensource.
All of our integrations aregoing to be but rsdks and and
our clients and all that, butthe core is not, and part of the
reason for that is, like opensource is very important to us.
We didn't want to make adecision to open something up
and then possibly pull that backlater.
Um, and we had some concernswith, you know, some of the
larger cloud providers and kindof what make a decision to open
(27:27):
something up and then possiblypull that back later.
And we had some concerns withsome of the larger cloud
providers and kind of whathappened with Elastic with their
license.
So we're like, well, let's notopen that door at this time, but
we still want people to be ableto use it and not have to pay.
So that's why we have it.
It's free to use.
You can download it.
We also have a free cloudhosted version, which we call
(27:51):
SearchRef Cloud.
That's got a free tier as well.
You have options as a developer.
You can either run it yourself,and that's totally fine, or you
can use our hosted option andhave a free tier.
That's fine too.
We really don't uh care likewhich way you go, as long as you
like, try it out.
Um, another reason why wedidn't open it up is, like it's
(28:13):
a very search, is a very likeniche problem, and we were going
to probably be the solecontributors to the repo anyway,
so it just really didn't makesense for us at this time.
Eyvonne (28:26):
So, from a product side , as you're thinking about
Searchcraft like what's your,what's your ideal use case, or
if somebody's thinking like isthis a good fit for me, like
what's what's kind of your, youknow the ideal either problem
you're solving for or situationin which Searchcraft's a perfect
fit.
Don (28:46):
Oh sure, yeah.
So unlike Elastic, we're nottrying to solve everyone's
problems.
So Elastic is really good at alot of different things, so like
log search, content search, etcetera, we're not trying to do
like log search specifically.
So if you're looking for a logsearch, you can hit me up.
I can make some recommendations, but don't use search craft for
that.
We are very focused on contentsearch, so human-oriented
(29:11):
natural language content, andthe reason I say that is we
optimize a lot of our featuresaround that and making sure that
you can turn on these featuresat query time and still have
very good performance for thingslike typo tolerance or fuzzy
matching and whatnot, whereassome of our competitors, you
know, if you want to turn on thefull suite of features, you
(29:33):
kind of have a trade-off.
We're like, okay, I'm going tohave a performance issue when
I'm getting in the millions ofdocuments.
Eyvonne (29:39):
Now, right, so very much the content providers,
media, long form content, thosethose kinds of arenas yeah, like
knowledge bases, documentationis great.
Don (29:51):
Uh, you know publishing content, like you know, news
organizations um, e-commerce islike.
This is kind of one of thethings.
William (30:13):
I was really super excited.
I mean a lot of it.
You know, a lot of the stuffwe've been talking about is
really relevant and exciting,but, um, just kind of the
current state of the market, um,like fundraising, for instance,
like you know, let's I guesslet's just talk about it every,
every founder is dealing with itright now fundraising is hard.
Um, the data is, you know,pretty stark.
(30:36):
Like, basically, unless itseems like, unless you have ai
prominently in your pitch, youhave ai coming out of your palms
and everything's AI it issignificantly harder to raise
capital in 2025.
And we're seeing anybody thatlooks at the news for more than
(30:58):
five minutes is seeing these AIcompanies raise money and it
just massive valuations with youknow where the math just
doesn't even make sense, youknow, while non, non AI
companies tend to struggle.
Do you have any experience hereand, if so, like?
What has been your experiencewith fundraising in this
(31:19):
environment, or what are yourthoughts?
Don (31:46):
no-transcript.
Understand that ai does notalways need to be the hammer
that drives on the elf, right,um, but there's a good number of
them that were really feelingthe pressure to make sure their
investments like have an aiangle to them, like from their
partners or whatever right.
So it's a question that was notasked so much at the beginning
(32:08):
of when we started our fundraisebut, like now and later on in
the process, it very much was um, so like our core product, um,
when we built it, did not relyon ai at all, uh, but the
landscape, as you say, hasshifted since we built that core
.
So, um, I think, um, you knowit's ai is a very transformative
(32:31):
technology.
It's fundamentally changing thelandscape of business, um, and
that has impacts on andeveryone's businesses.
Um, our view on ai is that, like, we're applying it where it
makes sense to kind of augmentand improve the overall
experience.
So, like AI is very good atapplying transformations or
analysis over an existing dataset, so we're utilizing it to
(32:55):
power, like some of those typesof features like trend analysis
and like generative dataenrichment.
We're also doing naturallanguage processing, semantic
search via our SDKs.
So that's going to be a layerwhere, if customers want that
kind of like transformative,like summary response, they can
(33:18):
use that and we'll have thatfeature available to them.
But if they don't want it, theydon't have to have it, because
some of our customers actuallylike, don't want ai like they've
.
They've said like we just want asmaller vaster elastic, we
don't want an ai feature and solike that's kind of why we don't
ship that as part of our core,but we do have it available in
both, like our cloud dashboardand our sdk, so like people can
(33:40):
pull it in and use it or theycan like opt out of it and not
um, but we're kind of a bigbeliever of choice.
So that's kind of like we'renot going to force that on
customers if they don't want it,but it is there if they do
gotcha.
William (33:54):
Yeah, that makes a lot of sense.
I really I could be justtotally wrong here.
But just an observation.
It seems like you have likereal the, you know the
foundational model companies,like companies that are actually
building ai, or they build, dothey procure and produce the
infrastructure that powers it,and then you have, like it's
(34:14):
almost like everybody else iskind of like using it or
incorporate, eitherincorporating it with like
existing features or making itas they go in and they pitch, or
they try to get funding, like,yeah, we have this awesome thing
, but we're using AI to enhanceit like X, y and Z, because,
let's face it, if you'rebuilding AI, there is a
(34:37):
difference between those twothings, I think.
Don (34:39):
Oh for sure, yeah, and these businesses are kind of
just like they're going to wrapsome llm or whatever, like do
what differentiates them from,like the other people who are
doing the same thing, right?
William (34:56):
um, I think a lot of those are going to fall away
after a while, after the hypekind of settles a bit yeah, and
I think that your, your responsethere is, I thought, pretty
thoughtful because it's like oneof you know, I just the media,
youtube everywhere is sosaturated with AI like, hey, you
can vibe code this startup andthat's just not the case.
(35:18):
Maybe one day it will be andvibe coding is a real thing, but
it's got its limits, of course.
But it's got its limits, ofcourse.
You know you still have to beable to validate and know and
like, look at what you're doingand you still need expertise.
Yvonne and I were actuallytalking about that this morning
um on, kind of like, the stateof that um in the market.
But you know, positioning likesearch craft in relation to the
(35:41):
AI wave, I think the kind of theway you pitched it was wasn't
disingenuous, it was hey, thisis, this is what it is and this
is how it's getting used.
Don (35:50):
So pretty, yeah, I mean pretty awesome as a tech company
, you do have to be able toanswer that question, because
people will ask you like okay,well, what about AI?
Why should I use your productif you don't have AI?
William (36:01):
so to check a box it's like the new, the new security
team.
Yeah, but yeah also like justbecause you produce an mcp
server doesn't mean that you'rean ai company, you know yes, I
mean we're not in the modelbusiness.
Don (36:19):
We don't want to be and like we actually don't prescribe
a specific model for, like, ourimplementation.
Uh, we let people can.
They can bring their own ifthey want.
I mean, we can like stand thefull thing up for them, but if
they want to bring their ownbecause they're already paying
for like mistral or clot orwhatever they can.
So so I think we still don'tknow yet who's gonna be like the
(36:42):
winners and losers on the modelside, right?
William (36:47):
yeah, definitely, um.
So I guess, like, as we nearthe end of the um the hour, um.
So what?
Just out of out of curiosity,what's your favorite developer
tool outside of search, or whatdo you like working with?
Don (37:04):
um, oh wow, let me, let me think here uh I was, I.
William (37:10):
I was actually asked this question the other day and
I froze.
Don (37:13):
I was like I don't know um, does it have to be okay?
Can it be like a, a thing youpay for, like a sass thing?
Is that okay?
Oh yeah, I'm a big fan ofPlausible.
I don't know if you guys haveused Plausible at all.
William (37:31):
Uh-uh.
Don (37:32):
So they're an analytics platform, but their whole thing
is not tracking people and verymuch anonymous analytic
reporting.
So we pretty much use them onall of our marketing sites that
we build, because it's just kindof less shady than the Google
(37:52):
Analytics and it also means youcan get by not having to do
those giant consent banners thatyou have to do for GDPR, which
is really nice, because I hatemaking those.
William (38:04):
Yeah, okay, next one.
What's your biggest technicalchallenge, looking back, that
you've overcome in building thenew uh product?
Don (38:18):
okay, um, for me it's like having to do a lot of the.
I mean, I I'm always buildingthings I haven't had to build
before, so at the beginning itwas like learning about how do I
build this core search library.
But once we got past thatbuilding up the whole tooling
(38:43):
and infrastructure around makinga software sas company um, we
have like internal, likeanalytics tracking for like what
people are searching on andthings like that, and they need
to be scalable in a way where wecould, um search across or like
track analytics across, likemultiple search index clusters,
(39:04):
but have it in a, a centraldatabase.
So we went with a productcalled ClickHouse.
I don't know if you guys arefamiliar with ClickHouse, but
the documentation is not great.
So, figuring out how to standthat up and scale that on our
AWS cloud, it took a while, andthen we'd put it up get it up in
(39:25):
production and we'd startgetting traffic into it.
And then we put it up, get up inproduction and we'd start
getting traffic into it and then, like we would have issues.
I'm like, and this thing, likeit, gets used by companies like
ebay and netflix and these hugecompanies.
So we know it's, it can scale.
We just don't know how to liketune it properly.
So there was quite a bit of alearning curve with that it
(39:50):
properly.
William (39:50):
So there was quite a bit of a learning curve with
that gotcha and I guess my lastlightning round question is if
you can give you know aspiringdevelopers that are just kind of
getting their you know, dippingtheir toes in the search um
arena, you know what would beone piece of advice, uh, that
you would give to them aboutchoosing search infrastructure.
Don (40:11):
Well, hopefully you don't have to become an expert in
search, because if you're havingto become an expert in search,
then I didn't do my job well.
So we want it to be anexperience where people can just
pull it off the shelf, get itup and running and be on the way
and build things that actuallymatter to their product, like
building out their productfeatures, not building search.
(40:31):
So it really should not besomething that people should
have to deal with like, um, justkind of like like
authentication, like you don'twant to be dealing with those
types of problems.
You need to deal with problemsthat make your product unique
and let, kind of like, theexperts deal with those other
issues.
Right, that's my advice.
(40:52):
Anyway, like I wouldn't if Iwas coming in, I now, like I
wouldn't want to learn all theyhave to learn to build search,
right, like, just do somethingoff the shelf.
William (41:04):
I can honestly say search and authentication,
identity and access managementare two areas where I just want
to off the shelf.
I can honestly say search andauthentication, identity and
access management are two areaswhere I just want to run and
hide.
Don (41:13):
Yeah, I've built out off systems and most people should
not do that.
They should absolutely not dothat work.
William (41:21):
Totally agree.
You got anything else, yvonne.
Eyvonne (41:24):
Before we wrap up no, I'm all good Love to know where
to find Dawn online and ifpeople want to know more where
they can look.
William (41:32):
Yeah, where can we find you?
Don (41:37):
So you know I'm on LinkedIn and Blue Sky primarily.
I'm not really on Twitteranymore, but if you go to
dawnmckinnondev it has links toboth Searchcraft and my LinkedIn
and my GitHub and my Boost.
William (41:51):
Awesome and I'll link that.
Yeah, I'll definitely link allthose, especially the GitHub,
and, yeah, everybody out therecheck it out and thank you so
much for coming on and weappreciate the time.
Popular Podcasts
Fudd Around And Find Out
UConn basketball star Azzi Fudd brings her championship swag to iHeart Women’s Sports with Fudd Around and Find Out, a weekly podcast that takes fans along for the ride as Azzi spends her final year of college trying to reclaim the National Championship and prepare to be a first round WNBA draft pick. Ever wonder what it’s like to be a world-class athlete in the public spotlight while still managing schoolwork, friendships and family time? It’s time to Fudd Around and Find Out!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.