August 5, 2024 • 41 mins
Have you ever stumbled upon an article or a piece of content online and wondered, "Did someone actually write this, or is it the work of ChatGPT?" In today’s world, where content is produced at an incredible pace, it's becoming increasingly difficult to tell the difference.. and that’s a problem in the age of misinformation.
Think about it: people are getting their news on social media, X, Youtube or Facebook! With the advancements of AI, it’s hard to tell how something online can be truly authentic. With latest studies showing >12% of Google's search results being AI-generated, it's critical to ensure the integrity of the digital content we consume and create.
That's where Originality AI comes in! We’re thrilled to host Jon Gillham, founder and CEO on Things Have Changed. as he shares how his team are tackling these issues head-on by developing cutting-edge tech to detect AI-generated content. In a short span of time, Originality AI have achieved remarkable results, and is the most accurate AI Detector in the market for ChatGPT, GPT-4o, Gemini Pro, Claude 3, Llama 3 etc.
So today on Things Have Changed, we'll dive deep into how Originality AI works, its impact on various industries, and why ensuring content authenticity is more important than ever.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
Today, The Washington Post andAxios reported a group of
leading tech companies,including Meta, Google, and
TikTok, committed to limitingmisleading A.
I.
content on their platforms.
Jed Tabernero (00:11):
Ever scroll through your newsfeed and paused
to question whether what you'rereading was created by a person.
Or a machine..
This question is becomingincreasingly relevant as AI
generated content floods, ourdigital spaces.
Making it harder to distinguish.
Between authentic human storiesfrom those fabricated by
(00:35):
algorithms.
Is there a concern thatmisinformation could rear its
ugly head and it would bedifficult for users to discern
what's real and what's not.
Today, we're super excited to have someone
who's been thinking about thisproblem for awhile.
John Gillham (00:52):
what we were building was a tool to help
people that were hiring writersknow whether or not they were
getting human written content orAI generated content.
And then, yeah, we ended uplaunching on Friday and then
basically the next Monday chatGPT launched
John Gilham, the CEO oforiginality.
AI joins us to discuss thetechnological arms race between
(01:14):
content creation.
And detection.
Join us on things have changedpodcast to uncover how
originality AI is shaping thefuture of digital authenticity.
Shikher Bhandary (02:02):
Surfing the web these days, I'm not sure if
an article or content that Icome across was actually written
by a human or by ChatGPT orClaude or any of the many AI
chatbots that are availablethese days.
So it's hard to tell thedifference and what makes it a
(02:26):
more pressing problem is we livein this age where things can go
viral in a minute and it couldjust straight up be wrong,
right?
We live in the age ofmisinformation.
So content is being produced atan incredible pace and we can't
verify the authenticity, right?
That's just like a recipe fordisaster.
(02:47):
People get their informationfrom Twitter now X or Facebook.
And with that, it's reallyimportant for us to find out if
certain pieces of content istruly authentic or not.
Right.
And that's where.
Our incredible guest today comesin.
We are super thrilled to haveJohn Gilliam, the founder and
(03:10):
CEO of Originality AI.
And John and his team aretackling these issues by
developing the technology toactually detect AI generated
content.
John Gillham (03:24):
Yeah, thanks.
Thanks for having me.
And yeah, I know it's we didn'tintend to build a societally
important product, but thatcertainly is is becoming what it
is.
So, yeah, no, it's a good to behere.
Shikher Bhandary (03:35):
yeah, even before jumping into the actual
platform, John, you have greatexpertise experience in this
space of just content, be itcreation, be it marketing, like,
how did that come about that youended up in this space of
actually detecting theoriginality of the content
(03:57):
itself.
John Gillham (03:58):
The journey was building.
So worked as an engineer and wasbuilding side businesses.
This 15 years now.
Thanks a lot of those businessesgenerated traffic by Google
publishing content on the webhad built up that business and
that's their portfolio, and thenhad a content marketing business
where we were doing that sameservice for others, where people
(04:19):
would hire us for gettingoptimized content created that
would rank well on Google.
And we were struggling to beable to communicate that there
was certainty that the writerhad created the content versus a
I had created the content.
And this was chatty.
We T changed the world in termsof the number of people that
were aware of generative a I.
But there was very big, verypopular tools that predated
(04:42):
Chachi PT that were built off ofopen a eyes.
GPT three.
And so that was what we werebuilding for was a tool to help
people that were hiring writersknow whether or not that they
were getting human writtencontent or AI generated content.
And then, yeah, we ended uplaunching on the Friday and then
basically the next Monday thatchat GPT launched obviously had
(05:03):
no idea that it was launchingand yeah, kind of change, change
things.
Shikher Bhandary (05:07):
Talk about a product market time fit.
John Gillham (05:12):
Yeah, yeah, it was when it first came out, I was
because we just spent monthstraining or a I'm being able to
detect GPT three a content andwe were like, Oh, no, is this a
new model?
But you know what?
We've all learned since is thatthis is just a, Wrapper with
some protection around anexisting open AI LLM.
(05:32):
So it was it didn't actuallydrop our performance, but it was
yeah, it was frustrating when itfirst happened at first.
And then we realized that itwas, I don't know, this is a
good thing.
Jed Tabernero (05:41):
John, could you kinda walk us through just an
example or a scenario where it'simportant to understand what is
AI generated versus what's humangenerated?
John Gillham (05:52):
Yeah, I think it's a question that it's a complex
question.
It's like society's gonna haveto wrestle with, over the
coming, coming decades that, youknow, where is it?
Okay.
Where's it?
Not?
Okay.
The students submitting piece oftheir paper.
We don't love a detection ortool being used within academia,
but I think that's a prettyclear one that like, no, that's
not your work unless you, youwrote it a.
(06:13):
A lot of our users use it thatthey're happy to pay a writer
100, 000 an article, whateverthe rate might be not super
happy to find out it was copiedand pasted out of chai GPT.
There's just that sort offairness component.
And then the most.
Publishers that are publishingcontent on the web would have a
similar view that I have thatpublishing AI generated content
(06:35):
introduces a new risk to yourwebsite in the eyes of Google.
And so a lot of publishers arewanting to know, to be the ones
that accept that risk, to knowwhere AI has been used and have
adequate sort of mitigation inplace to ensure that the other
downsides to AI content don'texist there.
So those are some of our usersfrom.
Us as like people using theInternet.
(06:56):
One of those main ones is whenwe're reading reviews.
I think it's pretty clear,society is going to have this
sort of debate about like whereis it okay for AI was not okay
for AI, but reading about areview for on baby formula.
And finding out that it was AIgenerated.
No one's, no one's okay withthat.
I would say is the sort of oneof those examples of that's a
(07:18):
pretty clear that society isnever going to be okay with AI
generated product reviews.
Jed Tabernero (07:23):
Sorry, I just wanted to clarify, did I hear
that correctly that AI generatedcontent is just ranked lower?
Do you know that an SEOs or isthat something that you
verified?
John Gillham (07:33):
We have some data that supports that, I think but
it's sort of questionable.
I think Google has made it veryclear that they are against spam
content.
And I think what's clear rightnow is that not all AI content
is spam, but all spam content isAI right now.
And so the result of that isthat Google has taken action
(07:53):
against sites and they did the,there's a March 5th manual
action.
Where they de indexed thousandsof sites when we looked at those
sites that have been de indexed,the majority of their content
was a I generated content, andso it doesn't it isn't
necessarily a one to one where aI content equals bad, but it is
a risk factor for publishers ifthey have a I content being mass
(08:19):
produced on their site, andespecially if they're paying
writers.
We had some websites come to usthat said our writers don't use
a I.
But we got a manual actionupdate on March 5th, and we were
able to look and say, You didn'tknow your writers were using AI,
but they were, and your websitehas now been de indexed and your
business is destroyed.
And AI content unmitigated is arisk for publishers.
Shikher Bhandary (08:44):
And this explosion is not going to stop
anytime soon, John, right?
Because we were reading thesearticles and you provided a lot
of context to where there areestimates that over 90 percent
of new content.
will be AI generated over thenext two, three years.
(09:05):
It's fast and it's explodingexponential growth from
literally two years ago.
John Gillham (09:12):
Yeah, it's crazy.
I'd say a couple interestingdata points 97.
5 percent of the internet Getszero clicks from Google.
So if Google sends basically 60percent of the world's traffic
flows through the pipes ofGoogle.
Currently, we'll see if thatchanges with generative item
that might and 97.
5 percent of the Internetdoesn't get a single click from
(09:32):
Google.
And so that's an interestingsort of the lens that is Google
is like the filter that isGoogle will filter out a lot of,
Likely a generated spam thatdoesn't deserve to be viewed and
doesn't add a lot of value intothe world.
We're tracking.
And so those are under our ownstudies.
Those are other studies.
We're tracking the amount of acontent that's showing up in
(09:53):
Google.
And it is Every month continuingto increase last update was 13
percent of the top 20 searchresults on across the sort of
sample of 200, 000 results wasshowing up as a suspected of
being a generated content andevery month it's increasing
already
Shikher Bhandary (10:12):
Wow.
Okay.
So the same study that I waslooking at, it was 2 percent a
year ago.
John Gillham (10:17):
13%.
Okay.
Shikher Bhandary (10:17):
We are looking at something that no one really
knows how to deal with.
Because it's so nascent and it'sso fast.
John Gillham (10:25):
Yeah, so it raises the question to have, from a
Google standpoint, like if thosesearch results become overrun
with AI, then why go to.
Cool.
Google and then like filterthrough links and then click on
an ad and have to read like abunch of content versus just go
to the AI that probably at thatpoint knows something about you
might be native in your device.
And would provide potentially amore beneficial answer.
(10:46):
So I think it's an interestingchallenge for Google to face to
be this sort of tech forward AIforward company while still
trying to deliver.
Search results.
It's their golden goose.
That is human.
If it's gone, then what's left?
Jed Tabernero (11:02):
I think it's interesting because the last
couple of years where we've seenthe blow up of generative AI,
right?
People have used itindiscriminately on everything,
right?
I got people using it for, I gotpeople, I used it for a wedding
invite, right?
(11:23):
That I just had to do quiterecently to go look it up,
right?
And it's interesting becauseBringing this specific topic up.
When we started doing ourresearch, we started thinking to
ourselves like, Oh, maybe it'snot super great to even use a
small portion of this AI.
Or we started thinking about whyit wouldn't be so great to use
(11:44):
AI for this, because initiallywe were like, Oh, this will take
care of 90 percent of ourcontent creation problems.
But now we're seeing that kindof.
There is a double edged sword tothat.
And this piece comes in.
That's why I asked the questioninitially to say, is it lower
ranked than other things, ordoes it hit your ranking if if
you were to generate AI, I thinkthat's just such an interesting
(12:05):
thing and people are going tostart realizing that there are
some issues with postingcompletely AI generated content.
The
John Gillham (12:13):
100 percent I think I'll a lot are and it
still shocks me at some that arejust learning that now that have
no like big publishers that haveno sort of a I policy in place,
let alone a I controls in place.
That it's a it's yes, it'spretty it has to your point, has
moved so fast that not allorganizations were capable of
(12:35):
moving at the pace that this is,this has moved at.
Shikher Bhandary (12:39):
Yeah.
John, it just, talking about anorganization that has moved so
quickly that they were two daysbefore they launched two days
before ChachiBD released.
So I've having used yourproduct.
It's amazing.
Gives me great understanding ofwhat the veracity of the
(13:02):
originality of the content thatI provided has.
Right.
So I'd love to understand theplatform that you have built and
maybe peel the layers a bit asto how it effectively works.
I think did mention aboutactually training the AI on a
previous GPT from open AI.
John Gillham (13:22):
Yeah, so the first step was, I fortunately was have
a great fortunate to hire a AIresearch lead that has now built
up a team with him.
But he's been phenomenal.
So the way the it can be kind ofunsettling.
Like a lot of things with whenit comes to AI and like, how
does it work?
What, why did this tag, why didthis document get identified as
AI generated?
And the answer is we don't know.
(13:43):
Because the model picked up somepatterns that are, Likely too
complex for us to think through.
It's certainly at that speed.
And so the way it works is it'sa classifier.
The simplest form of it is it'sa classifier.
That gets trained on at thispoint.
Millions of records of humanknown human content.
Millions of records of known AIacross all of the top.
(14:07):
Large language models.
So whether it be cloud three orGemini Pro Lama three or, of
course chat, GPT and GPT fourgets trained across all those
and then starts to learn to tellthe difference between what is
human and what is a I train upthe model and then run it
against a bunch of our ownbenchmark test.
And then at this point, nowthere's a bunch of publicly
(14:28):
available benchmark test that wecan run any new model against to
see what our Accuracy rate iswhat our false positive rate is.
And those are really the twonumbers that are our customers
care about the most is correctlyidentifying as a is a I
incorrectly identifying human ashuman.
Jed Tabernero (14:46):
Yeah, that's actually an insane task.
If you think about it.
I asked AI last night when I wasdoing the research to say, how
do you think something like thisis built?
How do you actually determinewhat the difference of a human
and AI generated content is?
If you think about it, we kindof notice.
Right.
I think me and you at leastchecker, when we go on LinkedIn,
(15:07):
there are some obvious poststhat are like, okay, this guy
Shikher Bhandary (15:11):
I'm thrilled to announce.
Jed Tabernero (15:14):
one's human.
That one's human.
Shikher Bhandary (15:15):
That was, that is human.
Yeah.
Jed Tabernero (15:18):
But it was interesting to learn about this
concept of stylometry.
Like a linguistic kind of styleand basically.
All the content that wasgenerated like pre a certain
date when generative AI waspopular was human generated
content, right?
So it's just for me, I can stillunderstand completely and very
(15:40):
clearly what AI writes for me,but I know that there's
something off about it and Ican't explain it.
And the way I read into kind ofthat research of how to
determine accuracy is there'sthat unknown sense that I have,
that this is AI generated.
Basically that was what was putinto your product and say,
(16:01):
inherently, this is what it'slooking like.
I think that's beautiful.
John Gillham (16:05):
Yeah.
What's So you talk about theability to identify that we
think we can identify a content,and there's been some not around
studies, but some interestingstudies that have been done on
this, and it's pretty shocking.
So I think, as humans, we haveto cognitive biases that can
greatly impact because I agreewith you, I think I can tell.
Even though I've seen thestudies but if we, if you were
(16:25):
to like ask a room, so like acouple of cognitive biases that
I think we are all susceptibleto.
If you were to ask a room onwho's an above average driver,
70 to 80 percent of their roomputs up their hand that they're
an above average driver.
Should be 50%.
But we have this overconfidencebias.
And then there's a patternrecognition bias that where
humans always are trying to makesense of the world.
(16:47):
And wherever there's chaos, wetry and recognize a pattern in
that chaos, ask any kind ofanyone in a casino.
No, I've got a system for thisrandom game of chance.
And in tests that looked at it.
Just sort of straight ability totell was this piece of content
human or this piece of contentAI with no sort of additional
controls like Here's thestudents ten pieces of previous
(17:10):
pieces of work and here's theirnew piece of work that was AI
generated There's a highprobability of being able to
tell But if it's just like thismight be human this might be AI
And what's humans ability totell the difference between the
two and it was barely betterthan flip of a coin as soon as
any adversarial technique wasput in place if it's just like
straight GPT content with noadversarial technique, just hey,
(17:30):
right X, then it writes in thatGPT for kind of style the chat
Shikher Bhandary (17:34):
Structured where no one speaks like that.
Yeah,
John Gillham (17:38):
yeah, but if you ask it like, right, like so and
so, humans ability to tell thedifference between AI and human
is totally out the window.
Flip a flip of a coin.
Shikher Bhandary (17:48):
got it.
And so John, this is just out ofmy curiosity.
It feels like with 4.
0 and maybe the latest updatefrom Cloud as well, I think more
on the Cloud than on the chatGPT side, it feels a bit more
human earlier iterations, likewhen GPD 3 and 4 came out, 3, 3.
(18:12):
5, I think.
Yeah.
It was easier to tell even fromlike someone who's seeing this
daily.
So you can pick up certain cuesin the text.
What happens in the case wherethese models get to the point
that they jumble up aspects ofhuman cues and.
(18:32):
AI structure to give you thismess that is so hard to figure
out and especially is probablyone of the biggest challenges
that your team faces, right?
John Gillham (18:43):
So for sure it is for sure it's it's a, one of our
biggest concerns, the data saysotherwise right now, which is
interesting in that are when soI think you're exactly right
that when, like when judge UBTfirst came out, it was hard to
make it right differently thanit's like forced style.
Like it was like, but then whenGPT four came out, You could
(19:04):
say, write a Nobel Peace Prizeaccepted speech in the style of
Dave Chappelle.
And no one would think that hadbeen AI generated pre that
written piece of content.
And so I think humans ability totell went out the window because
the diversity of what it couldcreate and the directions that
you could provide it became sosignificant.
What we have seen from Our ownmodel, whatever it's picking up
(19:27):
on whatever our own AI ispicking up on, we've seen the
new model will come out, ouraccuracy rate would drop from,
let's say it was 98 percentwould drop down to 85%, big,
unfortunate drop, have to trainup on that new model.
And then we would close thatgap.
What we have seen with thelatest models, basically no drop
off in 4.
0, Cloud 2 to Cloud 3 wasminimal drop off, Gemini Pro, no
(19:49):
drop off.
We're seeing a lot of thesemodels are trained on same data,
common crawl of the web, samehardware, same sort of
foundational technology aroundtransformers and we're seeing
this in 4.
0.
Diminishing a gap get opened upwith every new model over our
capability of detection.
That's what we're seeing rightnow.
(20:11):
Pretty hard to bet against theexponential growth, the
exponential amounts of moneythat are being poured into the
space.
And when GPT 10 comes out, arewe going to be able to do that?
Accurately detected.
I don't know, but I'd say ourcurrent sort of our gap to new
models has been closing with ourcurrent detection capability.
And I think that's leading to usbe on the side of okay, we're
(20:32):
seeing a little bit of aplateauing right now around
model capabilities.
And a right intelligence ofthese models.
The jump from GPT two to GPTthree felt bigger than the jump
from GPT three to GPT four.
Will this will the same exists?
Well, that will we see the samesort of Perceived diminished
(20:53):
jump on the next models.
I don't know.
Jed Tabernero (20:57):
Very interesting.
So using AI to detect AI, westarted looking into kind of the
other platforms in this spacejust to understand, Hey, what's
the competition looking like?
There's other models probablydoing a little bit of, the same
industry.
What would you say?
Sets yours apart than theseother companies and a lot of the
(21:21):
cool features I want to go overhere later But I'd love to hear
from your perspective you knowWhat sets you apart from the
other model
John Gillham (21:29):
Yeah, I think it's a two things and they're
related, but I think we madepretty quickly the decision that
like the world we understoodwhere was digital marketing
content being published on theweb.
That was the world that weunderstood.
Where we had felt like an unfairadvantage.
That's who we started buildingfor.
And so we build for basicallycopy editors that exist in any
organization that are getting apiece of content and then need
(21:52):
to publish that piece ofcontent.
So copy editors, toolkit, AIenabled toolkit.
And that's what we're building.
Core product is AI detection.
Because of that, because of whowe're building for, it's a far
more B2B than a B2C play.
Our free tier is far morelimited.
Our our pricing doesn't need tobe.
We're not competing forstudents.
(22:12):
And so because of that, we cangive more horsepower to our
detection team to say, run, runharder on the compute.
We can have a false positive todetection rate that is tuned to
our users use case and notneeding to be the sort of super
general detector.
And so because of all thosethings our data set gets tuned
(22:34):
to our users as opposed to beingthis sort of general purpose
detector.
And so I think that decision ofwho we're building for has led
to a bunch of other sort oftweaks along the way with the
model that has led to repeatedlyshowing up as the most accurate
aid detector.
AI research team incrediblysmart, but we're going up
against teams that are alsoincredibly smart I think the
(22:55):
combo of sort of that this AIresearch team being incredibly
good And then a bunch ofdecisions to get them aligned
with being capable of buildingGive them the tools to build the
best detector.
So most accurate detectors isone and then for our users,
which consistently proven bystudies.
And then the second being we'repretty clear on who we're very
(23:17):
clear on who we're building for.
Shikher Bhandary (23:19):
Big kudos to the fact that y'all are the most
accurate.
AI detection tool in the marketright now.
So that is in itself incredible.
So the stakeholders right now,the customers that y'all are
actively working with hand inhand, not just copy editors,
(23:40):
like you mentioned, but also,these marketing companies, maybe
educational institutions, likehow are you thinking through the
actual Customer cohorts totarget and build those
relationships with.
John Gillham (23:53):
Yeah, so we, I think we're, I think we're the
only tool that actively liststhat we're not for academia.
Shikher Bhandary (23:59):
Okay.
John Gillham (24:00):
that I think, so we do have a lot of academia
that uses us because we are themost accurate.
The sort of amount of toolingthat you need to build around
academia to be confident it'sbeing used.
In the right way is more than wethat's not the problem.
Like it's a big problem.
It's just not the problem.
We're focused on.
And so we know academia is usingus.
We don't love it being used foracademia.
(24:21):
So we false positives do happen.
And the amount of tooling youbuild around that to deal with
that within an academic settingis different than what we build
within a for for writers wherewe can have a free chrome
extension and some other toolingthat helps deal with false
positives in the writer.
Editor relationship.
And so that's like the one sortof unique condition for us.
(24:42):
And then the rest of our usersmostly fall into the digital
marketing world.
So web publishing world,whatever we want to call it, but
getting content from a writer.
Reviewing the content,publishing on the web, and so
any company where a copy editor,where somebody functions as a
copy editor then our tool can beuseful, and that can be
incredibly small companies, oneperson operations that has one
(25:05):
writer for their one website,and it can be incredibly large
organizations that havethousands of writers and
hundreds of editors.
Really focused on that role, orpeople functioning in that role.
Shikher Bhandary (25:16):
Yeah.
And it kind of fits, it probablyfits better to what your
incredible expertise is in anyway, right?
Because you've built what is it?
Three companies within thecontent space.
So you probably know exactly theworkflows exactly where in that
workflow and those individualsmaking these decisions.
John Gillham (25:38):
For sure.
Off the start.
I've been, I would have agreedwith you wholeheartedly a year
and a half ago.
I think after enoughconversations, I've been pretty
humbled at Oh, maybe I didn'tunderstand this space.
I didn't understand all parts ofthis space as well as I thought
I had.
The use case for somebody in alarge organization will be
different than a marketingagency, which would be different
than a website.
So in general, yes.
(25:59):
But I think Yeah, still lots tolearn on exactly how everyone's
workflow.
Even though it's a similarfunction works, but it is
definitely the space that weunderstand the best.
Jed Tabernero (26:09):
is there a specific feature that you're
most excited about John?
Of what you're providing todaythat be B to C or B to B
customers that you have Not
John Gillham (26:20):
This would be a weird answer.
It's the one I'm mostdisappointed in, but I'm also
most excited for.
So we built a fact checker soheavily reg enabled.
Fact checker, super intensive interms of going out, finding the
information for every statementof fact, trying to do reg,
laying it in, overlaying it withan LLM to then provide an answer
(26:41):
on, is this factual or not?
It's not very good.
So we're really excited for it.
It's still in beta.
It provides it's an aid in termsof providing research to help
people in the process of factchecking when they get a piece
of content.
I think hallucinations andfactual accuracy of what
(27:02):
elements output.
Is a pretty massive problem.
That is hard to solve by justenriching, increasing the data
because the so the unbalancedthe creative nature of these
models really, really hard tokeep them within.
Within parameters and factualaccuracy, even with all with a
ton of constraints put on uponthem.
(27:23):
So that's the one I'm excitedfor it because I think it's
solvable eventually.
And so we're keeping a reallyclose eye on when can this added
effort that we can Inject interms of reg and understanding
the web and trusted sources.
When can that be matched up withan LLM that will provide that
(27:44):
level of fact checking that wecan look at and be 99 percent
plus confident is accurate andwe're not there yet.
Jed Tabernero (27:53):
I think I'll just share what my favorite feature
right now is.
It's the readability piece, howcleanly it comes out when you
put something in.
Which I think if you guys haveused other tools like Grammarly,
it's it's in the same light.
But I think the fact that.
Right underneath the output.
First of all, you color all ofthe, the unreadable sentences,
the long sentences.
(28:14):
It just basically told me howshit of a writer I am.
But it's really
Shikher Bhandary (28:17):
I've been telling that to you like for two
years, dude.
Jed Tabernero (28:21):
it'll tell you something's you've got over 20
syllables in this sentence,right?
Yeah, that's difficult.
That's probably difficult tocapture.
And you don't realize thesethings when you're writing
stuff.
So I can see how it was builtfor those content marketers as
well, because these are thestuff, of course, me and Chick
are having this as a passionproject, this is one of the
things that we care about themost.
So very interesting.
(28:42):
You also have a feature onparaphrasing, which I think was
difficult for me to understand,but that's trained on something,
right?
Trained on a tool that, thatdoes the paraphrasing for you.
John Gillham (28:54):
Yeah.
This, what, one of the, one ofthe most fun aspects of this
role has been the cat and mousegame.
And just the, like the constantbattles of like launch detector.
And I was too dumb to listen tothe feedback that we got, but it
was like the feedback we gotfrom day one was is there a
button that I can just pressthat will make it past the
(29:15):
detection?
And I'm like, yeah, No, we can'tbuild that like our tool would
be useless with that which iswhich is true.
But I got that feed that requestso many times that I was like,
well, why didn't I see coming?
What came next, which is nowthere's all of these tools that
attempt to bypass us.
And so one of the most commonways, especially early on to buy
(29:36):
and still works for a lot ofother detectors was to use a
paraphrasing tool.
So using and that produces a newpattern.
And so you'll it'll You cancreate content using an LLM and
then remove, I won't watermarks,not the right term because that
has a whole other meaning to it,but remove the pattern
recognition that comes byparaphrasing.
(29:56):
And so that's, that was one ofthe earliest methods of trying
to bypass detection.
There's been a bunch more since,but there, yeah, anyway it's fun
to, we have a red team and ablue team.
Red team's always trying to beatour detector, find out what
tools are available to beat ourdetector and then build a data
set to try and learn against it.
Jed Tabernero (30:12):
I love that, that that's part of the culture.
There's a red team and a blueteam.
John Gillham (30:17):
Yeah.
Yeah, it's it's it's unique andfun for sure.
Jed Tabernero (30:20):
no, it makes it a lot more fun when I saw these
tools, just.
Thinking about somebody likemyself, who's in this space
where we're writing a lot ofstuff, we're talking about a lot
of things and then transcribing,then writing a lot of writing.
I thought to myself, this wouldbe really dope to integrate with
some of the native tools that wealready have, right?
Some of the workflows that wemight have in our space to just
(30:42):
see, Oh, you know what, let'sjust, when we write something,
make sure it goes through thistest to ensure that it's
readable, so that my shitwriting doesn't get published.
But, things like that.
A lot of people.
Are now publishing onlinecontent, as we said, and using
AI to do so.
So it's just like really useful.
Just question for you, anyintegrations in the future,
anybody courting you to become,part of their tool, et cetera
(31:05):
would love to hear about anyfuture plans like that.
John Gillham (31:09):
Yeah.
So we have a chrome extension.
That works that is tightlyintegrated into Google document
workflow.
Not so much from a creationstandpoint.
And so for some, to some extent,like what grammar really is to
writers, we are aiming to be toeditors.
And so if there's integrationsthat make sense on under that
lens, then yes, I don't think wewill aim to be.
(31:30):
Writing aid.
And so I think the level ofintegrations that occur if we
were focused on being a writingaid increases with a current
focus not opposed tointegrations, but they got to
make sense from that use case ofa copy editor.
And in those cases like we havea Chrome extension that is very
helpful for people to understandthe originality authenticity of
(31:52):
a Google document.
Shikher Bhandary (31:55):
That's great.
The way the media or the wayeven the folks in the industry
compare models is through, abunch of these.
technical jargon, right?
Tokens, parameters.
Oh, this is this size.
This is this size.
So specifically on your product,does it matter how advanced or
(32:17):
how many tokens or parameters ithas to be able to detect
something that was generated bya model that does have those
extraordinary numbers of tokensor parameters?
Is it something that you'rethinking through?
John Gillham (32:31):
Similar that sort of earlier conversation around,
like we're seeing this sort ofdiminishing,
Shikher Bhandary (32:37):
Okay.
John Gillham (32:37):
gap that is showing up with every new model
that comes out.
You know, exponential growth ofparameters.
Output is.
Better in some ways.
But then our capability ofdetecting it doesn't drop off.
And I'm hesitant to like projectout in this world.
What, what's coming, but I cansay what's happened, his history
being the last two years.
(32:58):
What we've seen is that thissort of exponential growth Of
parameters, model sizes,training costs data consumption
has not led to to the abilityfor these LLMs to create content
that whatever our models, ourdetectors, most accurate, but
there's other detectors that aredecently accurate.
Whatever they're these detectorsare capable of picking up is
(33:20):
staying is staying in placeright now.
And so we're seeing, I'd saywhere it's been interesting and
the biggest sort of gap that hasexposed.
There's all these differentcriterias for these.
for these models.
Some of the open source modelshave been, even though they
might have a smaller parametercount, smaller training size
they have produced someinteresting variability in, in
(33:43):
accuracy.
So like Mistral in particulardid, again, easy enough to
close, but that was one of oureven though most open source
models up to that point had notbeen I challenged I've been
quite easy to detect.
Shikher Bhandary (33:55):
It threw a wrench into things.
Yeah.
John Gillham (33:57):
yeah.
Yeah.
So that was interesting.
Shikher Bhandary (33:59):
Got it.
Yeah.
I was just thinking because thetalk of the town is LLMs, SLMs,
action models, and things likethat.
And I was just wondering wherein that whole category does an
AI detection for those LLMsactually sit.
But it's great that now thatyou're up 4.
0 or 4.
0 from here, it still isaccurate on a high percentage
(34:25):
basis.
John Gillham (34:26):
Yes, we're the most accurate we have ever been
on the latest model.
Our gap has been closing.
This could aged extremelypoorly, right?
GPT five could come out and makedetection totally impossible.
Oh, yeah, we'll see.
Shikher Bhandary (34:39):
you're doing your best.
You're doing your best.
Jed Tabernero (34:42):
Yeah.
Can you imagine if GPT 5 cameout and that's its entire goal
was to consume AI detectionsoftware?
Oh my god.
John Gillham (34:51):
You know, I just listened to Mira the CTO at
OpenAI talking about detectionand I, I think I understand why
they're detector.
So they had launched their owndetector, but they were given a
certain, like it was had to befree, had to be super low false
positive.
The result was pretty muchuseless on detection.
And so I understand and viewedwith a lens of certainty.
So I understood why.
(35:11):
Given the criteria that wouldhave been provided to their team
to build a detector, they wouldhave had to fail.
And I think they don't want, Ithink they would want their
content to continue to becapable of being detected
because It helps to decrease thesocietal harm that their product
will be capable of producing.
And that's at least the wordsthat they're saying.
(35:32):
They would love to havewatermarking.
I think watermarking will neverbe a solution within text.
But yeah, so that's what I wouldsay is it's interesting to yeah
be trying to read the tea leavesof what OpenAI truly cares about
moving forward.
Jed Tabernero (35:49):
Interesting point on the societal risk.
We touched on it in thebeginning, right?
To say that, hey, what's thescenario this actually might be
hurtful.
And we looked into someexamples, one of which was like
a book on Amazon.
That had very questionableinformation about foraging
mushrooms is it was, that wasyour bullet, right?
(36:10):
Chikor,
Shikher Bhandary (36:11):
Yeah.
Yeah.
And it was actually a link thatJohn shared and baby formula
stuff was also there where nowpeople are questioning.
Jed Tabernero (36:20):
because you, you don't think about these right as
a risk.
And I'll tell you why thisspecific example was so funny to
me in the beginning was becausewhen ChadCPT first came out, I
messaged Chikor and I said,look, I found another way to
make money.
I'm going to make a book and I'mgoing to have chat GPT write me
every chapter, to make thisbook.
(36:40):
And it was just interesting tome that the example that was
provided was a
Shikher Bhandary (36:44):
Literally someone who thought about that.
Jed Tabernero (36:46):
AI.
Yeah.
Literally someone who thoughtabout that and then put it on
there and sold, probably made alittle bit of money out of it.
People don't think about thatsocietal cost when it comes to
these like super useful thingsfor humans.
So the way that I think aboutoriginality AI in general are
guardrails.
To these, really crazyinnovations.
That's why I think it's sointeresting that we covered just
(37:08):
a company before thishighlighting, the power of what
AI can do and AI agents.
We talked a lot about AI agents,and we've been talking about a
lot of the positives ofartificial intelligence.
And I.
I appreciate that for thisepisode, we're actually able to
step back and say, look, this isone thing what we're doing with
an ad using AI actually toreduce societal, harm, which is
(37:31):
pretty awesome.
Have people who are talking toyou about originally AI over
index on the piece that you arereducing societal harm.
John Gillham (37:38):
I think there is a societal importance to the
company that, that I didn'tinitially set out to, to build,
and it is being used in waysthat I hope is reducing harm.
Again, a lot of our focus is onthe, that were the, the specific
vertical we're in, and in, inthat vertical, I think there's a
significant misunderstanding inthat space, where right now we
have writers, You need to useour platform to make sure their
(38:01):
content passes AI.
This is a problem that theynever experienced before.
They aren't using AI.
We have false positives.
And there's this sort of harmthat comes from that because
there's writers that aren'tgetting paid because there's a
false positive.
What I think is being missed isthat short of having some AI
Detection in that workflow thatthe volume of writers goes up
(38:25):
to, the, basically the worldpopulation that's connected to
the internet for a good chunk ofwriters.
ChadGBT writes better than I do.
Sounds like better than you,better than yourself.
But
Jed Tabernero (38:34):
Just me specifically.
John Gillham (38:37):
yeah.
So if I was, if I made mylivelihood as a writer AI
detection, although theoccasional false So that can be
harmful.
It's a, it's at least defendingthat industry from being totally
wiped out by by chat GPT andother LLMs.
Shikher Bhandary (38:54):
There's a lens or there's a view of, Hey, if
this is good for the consumer,why should they care?
And this is why they shouldactually care.
Right.
Because people are just going tobe like, Hey, Content is going
to be ingested regardless,whether it's from actual human
or AI, if it's good content.
What's the harm, right?
And there's actually a harm herebecause it just leads to other
(39:16):
things.
John Gillham (39:17):
Yeah, exactly.
Shikher Bhandary (39:18):
Just wrapping up, John, this was fantastic.
When we have such guests on suchfounders, academics on, we give
them the stage to give a shoutout.
Maybe it's for a team, maybeit's for a new product release.
Maybe it's fundraising too,because we have a lot of
founders that have actually Usedour platform to connect with the
right VCs and stuff.
(39:38):
We'd love to give you the stage.
John Gillham (39:40):
Yeah, no sounds good.
But yeah, so I'd say if anyoneis working in an organization
that, that has peoplefunctioning as a copy editor and
is trying to wrestle with thesequestions that, that we wrestled
with today on what is allowable,not allowable, and whether or
not the risks are, Adequatelymanaged in your organization
with the use of a I inspecifically for us within the
(40:00):
world of content marketing.
We're happy to chat and happy tohelp people think through the
appropriate uses of it and theappropriate sort of use of
originality for mitigating thoserisks.
Jed Tabernero (40:12):
Awesome.
And do people still get Freetokens.
If you get the Chrome extension.
John Gillham (40:17):
Yeah.
So if you sign up and and get,yeah, 50, 50 free credits, so we
have the free tool that you canuse which is super limited, and
then you can get 50 free creditsto the premium tool when you
confirm your, sign up, know withthe Chrome extension.
Jed Tabernero (40:29):
Sweet.
Well, this was super awesome,John.
We learned a lot, even justdoing the research, honestly,
but talking to you was a lotbetter.
Really appreciate your time andthanks for coming on.
Things have changed, man.
John Gillham (40:39):
Yeah, thanks for having me.
Fun fun conversation.
Thanks for tuning into today'sepisode of things have changed
podcast.
We hope you found our discussionwith John Gill, him
enlightening, and that itsparked new thoughts about the
digital content we interact withdaily.
Remember.
In a world where technologycontinually evolves, staying
(41:00):
informed is key to navigatingthe complexities of digital
authenticity.
And that's what urgent Lottie AIcan help you with.
Cheers.
And as always stay curious.
The views and opinions expressedin this podcast are those of the
guests and do not necessarilyreflect the official policy or
position of things have changedpodcasts or its affiliates.
(41:23):
The content provided is forinformational purposes only and
should not be taken asprofessional advice.
Listener discretion is advised.
Today, The Washington Post andAxios reported a group of
leading tech companies,including Meta, Google, and
TikTok, committed to limitingmisleading A.
I.
content on their platforms.
Jed Tabernero (00:11):
Ever scroll through your newsfeed and paused
to question whether what you'rereading was created by a person.
Or a machine..
This question is becomingincreasingly relevant as AI
generated content floods, ourdigital spaces.
Making it harder to distinguish.
Between authentic human storiesfrom those fabricated by
(00:35):
algorithms.
Is there a concern thatmisinformation could rear its
ugly head and it would bedifficult for users to discern
what's real and what's not.
Today, we're super excited to have someone
who's been thinking about thisproblem for awhile.
John Gillham (00:52):
what we were building was a tool to help
people that were hiring writersknow whether or not they were
getting human written content orAI generated content.
And then, yeah, we ended uplaunching on Friday and then
basically the next Monday chatGPT launched
John Gilham, the CEO oforiginality.
AI joins us to discuss thetechnological arms race between
(01:14):
content creation.
And detection.
Join us on things have changedpodcast to uncover how
originality AI is shaping thefuture of digital authenticity.
Shikher Bhandary (02:02):
Surfing the web these days, I'm not sure if
an article or content that Icome across was actually written
by a human or by ChatGPT orClaude or any of the many AI
chatbots that are availablethese days.
So it's hard to tell thedifference and what makes it a
(02:26):
more pressing problem is we livein this age where things can go
viral in a minute and it couldjust straight up be wrong,
right?
We live in the age ofmisinformation.
So content is being produced atan incredible pace and we can't
verify the authenticity, right?
That's just like a recipe fordisaster.
(02:47):
People get their informationfrom Twitter now X or Facebook.
And with that, it's reallyimportant for us to find out if
certain pieces of content istruly authentic or not.
Right.
And that's where.
Our incredible guest today comesin.
We are super thrilled to haveJohn Gilliam, the founder and
(03:10):
CEO of Originality AI.
And John and his team aretackling these issues by
developing the technology toactually detect AI generated
content.
John Gillham (03:24):
Yeah, thanks.
Thanks for having me.
And yeah, I know it's we didn'tintend to build a societally
important product, but thatcertainly is is becoming what it
is.
So, yeah, no, it's a good to behere.
Shikher Bhandary (03:35):
yeah, even before jumping into the actual
platform, John, you have greatexpertise experience in this
space of just content, be itcreation, be it marketing, like,
how did that come about that youended up in this space of
actually detecting theoriginality of the content
(03:57):
itself.
John Gillham (03:58):
The journey was building.
So worked as an engineer and wasbuilding side businesses.
This 15 years now.
Thanks a lot of those businessesgenerated traffic by Google
publishing content on the webhad built up that business and
that's their portfolio, and thenhad a content marketing business
where we were doing that sameservice for others, where people
(04:19):
would hire us for gettingoptimized content created that
would rank well on Google.
And we were struggling to beable to communicate that there
was certainty that the writerhad created the content versus a
I had created the content.
And this was chatty.
We T changed the world in termsof the number of people that
were aware of generative a I.
But there was very big, verypopular tools that predated
(04:42):
Chachi PT that were built off ofopen a eyes.
GPT three.
And so that was what we werebuilding for was a tool to help
people that were hiring writersknow whether or not that they
were getting human writtencontent or AI generated content.
And then, yeah, we ended uplaunching on the Friday and then
basically the next Monday thatchat GPT launched obviously had
(05:03):
no idea that it was launchingand yeah, kind of change, change
things.
Shikher Bhandary (05:07):
Talk about a product market time fit.
John Gillham (05:12):
Yeah, yeah, it was when it first came out, I was
because we just spent monthstraining or a I'm being able to
detect GPT three a content andwe were like, Oh, no, is this a
new model?
But you know what?
We've all learned since is thatthis is just a, Wrapper with
some protection around anexisting open AI LLM.
(05:32):
So it was it didn't actuallydrop our performance, but it was
yeah, it was frustrating when itfirst happened at first.
And then we realized that itwas, I don't know, this is a
good thing.
Jed Tabernero (05:41):
John, could you kinda walk us through just an
example or a scenario where it'simportant to understand what is
AI generated versus what's humangenerated?
John Gillham (05:52):
Yeah, I think it's a question that it's a complex
question.
It's like society's gonna haveto wrestle with, over the
coming, coming decades that, youknow, where is it?
Okay.
Where's it?
Not?
Okay.
The students submitting piece oftheir paper.
We don't love a detection ortool being used within academia,
but I think that's a prettyclear one that like, no, that's
not your work unless you, youwrote it a.
(06:13):
A lot of our users use it thatthey're happy to pay a writer
100, 000 an article, whateverthe rate might be not super
happy to find out it was copiedand pasted out of chai GPT.
There's just that sort offairness component.
And then the most.
Publishers that are publishingcontent on the web would have a
similar view that I have thatpublishing AI generated content
(06:35):
introduces a new risk to yourwebsite in the eyes of Google.
And so a lot of publishers arewanting to know, to be the ones
that accept that risk, to knowwhere AI has been used and have
adequate sort of mitigation inplace to ensure that the other
downsides to AI content don'texist there.
So those are some of our usersfrom.
Us as like people using theInternet.
(06:56):
One of those main ones is whenwe're reading reviews.
I think it's pretty clear,society is going to have this
sort of debate about like whereis it okay for AI was not okay
for AI, but reading about areview for on baby formula.
And finding out that it was AIgenerated.
No one's, no one's okay withthat.
I would say is the sort of oneof those examples of that's a
(07:18):
pretty clear that society isnever going to be okay with AI
generated product reviews.
Jed Tabernero (07:23):
Sorry, I just wanted to clarify, did I hear
that correctly that AI generatedcontent is just ranked lower?
Do you know that an SEOs or isthat something that you
verified?
John Gillham (07:33):
We have some data that supports that, I think but
it's sort of questionable.
I think Google has made it veryclear that they are against spam
content.
And I think what's clear rightnow is that not all AI content
is spam, but all spam content isAI right now.
And so the result of that isthat Google has taken action
(07:53):
against sites and they did the,there's a March 5th manual
action.
Where they de indexed thousandsof sites when we looked at those
sites that have been de indexed,the majority of their content
was a I generated content, andso it doesn't it isn't
necessarily a one to one where aI content equals bad, but it is
a risk factor for publishers ifthey have a I content being mass
(08:19):
produced on their site, andespecially if they're paying
writers.
We had some websites come to usthat said our writers don't use
a I.
But we got a manual actionupdate on March 5th, and we were
able to look and say, You didn'tknow your writers were using AI,
but they were, and your websitehas now been de indexed and your
business is destroyed.
And AI content unmitigated is arisk for publishers.
Shikher Bhandary (08:44):
And this explosion is not going to stop
anytime soon, John, right?
Because we were reading thesearticles and you provided a lot
of context to where there areestimates that over 90 percent
of new content.
will be AI generated over thenext two, three years.
(09:05):
It's fast and it's explodingexponential growth from
literally two years ago.
John Gillham (09:12):
Yeah, it's crazy.
I'd say a couple interestingdata points 97.
5 percent of the internet Getszero clicks from Google.
So if Google sends basically 60percent of the world's traffic
flows through the pipes ofGoogle.
Currently, we'll see if thatchanges with generative item
that might and 97.
5 percent of the Internetdoesn't get a single click from
(09:32):
Google.
And so that's an interestingsort of the lens that is Google
is like the filter that isGoogle will filter out a lot of,
Likely a generated spam thatdoesn't deserve to be viewed and
doesn't add a lot of value intothe world.
We're tracking.
And so those are under our ownstudies.
Those are other studies.
We're tracking the amount of acontent that's showing up in
(09:53):
Google.
And it is Every month continuingto increase last update was 13
percent of the top 20 searchresults on across the sort of
sample of 200, 000 results wasshowing up as a suspected of
being a generated content andevery month it's increasing
already
Shikher Bhandary (10:12):
Wow.
Okay.
So the same study that I waslooking at, it was 2 percent a
year ago.
John Gillham (10:17):
13%.
Okay.
Shikher Bhandary (10:17):
We are looking at something that no one really
knows how to deal with.
Because it's so nascent and it'sso fast.
John Gillham (10:25):
Yeah, so it raises the question to have, from a
Google standpoint, like if thosesearch results become overrun
with AI, then why go to.
Cool.
Google and then like filterthrough links and then click on
an ad and have to read like abunch of content versus just go
to the AI that probably at thatpoint knows something about you
might be native in your device.
And would provide potentially amore beneficial answer.
(10:46):
So I think it's an interestingchallenge for Google to face to
be this sort of tech forward AIforward company while still
trying to deliver.
Search results.
It's their golden goose.
That is human.
If it's gone, then what's left?
Jed Tabernero (11:02):
I think it's interesting because the last
couple of years where we've seenthe blow up of generative AI,
right?
People have used itindiscriminately on everything,
right?
I got people using it for, I gotpeople, I used it for a wedding
invite, right?
(11:23):
That I just had to do quiterecently to go look it up,
right?
And it's interesting becauseBringing this specific topic up.
When we started doing ourresearch, we started thinking to
ourselves like, Oh, maybe it'snot super great to even use a
small portion of this AI.
Or we started thinking about whyit wouldn't be so great to use
(11:44):
AI for this, because initiallywe were like, Oh, this will take
care of 90 percent of ourcontent creation problems.
But now we're seeing that kindof.
There is a double edged sword tothat.
And this piece comes in.
That's why I asked the questioninitially to say, is it lower
ranked than other things, ordoes it hit your ranking if if
you were to generate AI, I thinkthat's just such an interesting
(12:05):
thing and people are going tostart realizing that there are
some issues with postingcompletely AI generated content.
The
John Gillham (12:13):
100 percent I think I'll a lot are and it
still shocks me at some that arejust learning that now that have
no like big publishers that haveno sort of a I policy in place,
let alone a I controls in place.
That it's a it's yes, it'spretty it has to your point, has
moved so fast that not allorganizations were capable of
(12:35):
moving at the pace that this is,this has moved at.
Shikher Bhandary (12:39):
Yeah.
John, it just, talking about anorganization that has moved so
quickly that they were two daysbefore they launched two days
before ChachiBD released.
So I've having used yourproduct.
It's amazing.
Gives me great understanding ofwhat the veracity of the
(13:02):
originality of the content thatI provided has.
Right.
So I'd love to understand theplatform that you have built and
maybe peel the layers a bit asto how it effectively works.
I think did mention aboutactually training the AI on a
previous GPT from open AI.
John Gillham (13:22):
Yeah, so the first step was, I fortunately was have
a great fortunate to hire a AIresearch lead that has now built
up a team with him.
But he's been phenomenal.
So the way the it can be kind ofunsettling.
Like a lot of things with whenit comes to AI and like, how
does it work?
What, why did this tag, why didthis document get identified as
AI generated?
And the answer is we don't know.
(13:43):
Because the model picked up somepatterns that are, Likely too
complex for us to think through.
It's certainly at that speed.
And so the way it works is it'sa classifier.
The simplest form of it is it'sa classifier.
That gets trained on at thispoint.
Millions of records of humanknown human content.
Millions of records of known AIacross all of the top.
(14:07):
Large language models.
So whether it be cloud three orGemini Pro Lama three or, of
course chat, GPT and GPT fourgets trained across all those
and then starts to learn to tellthe difference between what is
human and what is a I train upthe model and then run it
against a bunch of our ownbenchmark test.
And then at this point, nowthere's a bunch of publicly
(14:28):
available benchmark test that wecan run any new model against to
see what our Accuracy rate iswhat our false positive rate is.
And those are really the twonumbers that are our customers
care about the most is correctlyidentifying as a is a I
incorrectly identifying human ashuman.
Jed Tabernero (14:46):
Yeah, that's actually an insane task.
If you think about it.
I asked AI last night when I wasdoing the research to say, how
do you think something like thisis built?
How do you actually determinewhat the difference of a human
and AI generated content is?
If you think about it, we kindof notice.
Right.
I think me and you at leastchecker, when we go on LinkedIn,
(15:07):
there are some obvious poststhat are like, okay, this guy
Shikher Bhandary (15:11):
I'm thrilled to announce.
Jed Tabernero (15:14):
one's human.
That one's human.
Shikher Bhandary (15:15):
That was, that is human.
Yeah.
Jed Tabernero (15:18):
But it was interesting to learn about this
concept of stylometry.
Like a linguistic kind of styleand basically.
All the content that wasgenerated like pre a certain
date when generative AI waspopular was human generated
content, right?
So it's just for me, I can stillunderstand completely and very
(15:40):
clearly what AI writes for me,but I know that there's
something off about it and Ican't explain it.
And the way I read into kind ofthat research of how to
determine accuracy is there'sthat unknown sense that I have,
that this is AI generated.
Basically that was what was putinto your product and say,
(16:01):
inherently, this is what it'slooking like.
I think that's beautiful.
John Gillham (16:05):
Yeah.
What's So you talk about theability to identify that we
think we can identify a content,and there's been some not around
studies, but some interestingstudies that have been done on
this, and it's pretty shocking.
So I think, as humans, we haveto cognitive biases that can
greatly impact because I agreewith you, I think I can tell.
Even though I've seen thestudies but if we, if you were
(16:25):
to like ask a room, so like acouple of cognitive biases that
I think we are all susceptibleto.
If you were to ask a room onwho's an above average driver,
70 to 80 percent of their roomputs up their hand that they're
an above average driver.
Should be 50%.
But we have this overconfidencebias.
And then there's a patternrecognition bias that where
humans always are trying to makesense of the world.
(16:47):
And wherever there's chaos, wetry and recognize a pattern in
that chaos, ask any kind ofanyone in a casino.
No, I've got a system for thisrandom game of chance.
And in tests that looked at it.
Just sort of straight ability totell was this piece of content
human or this piece of contentAI with no sort of additional
controls like Here's thestudents ten pieces of previous
(17:10):
pieces of work and here's theirnew piece of work that was AI
generated There's a highprobability of being able to
tell But if it's just like thismight be human this might be AI
And what's humans ability totell the difference between the
two and it was barely betterthan flip of a coin as soon as
any adversarial technique wasput in place if it's just like
straight GPT content with noadversarial technique, just hey,
(17:30):
right X, then it writes in thatGPT for kind of style the chat
Shikher Bhandary (17:34):
Structured where no one speaks like that.
Yeah,
John Gillham (17:38):
yeah, but if you ask it like, right, like so and
so, humans ability to tell thedifference between AI and human
is totally out the window.
Flip a flip of a coin.
Shikher Bhandary (17:48):
got it.
And so John, this is just out ofmy curiosity.
It feels like with 4.
0 and maybe the latest updatefrom Cloud as well, I think more
on the Cloud than on the chatGPT side, it feels a bit more
human earlier iterations, likewhen GPD 3 and 4 came out, 3, 3.
(18:12):
5, I think.
Yeah.
It was easier to tell even fromlike someone who's seeing this
daily.
So you can pick up certain cuesin the text.
What happens in the case wherethese models get to the point
that they jumble up aspects ofhuman cues and.
(18:32):
AI structure to give you thismess that is so hard to figure
out and especially is probablyone of the biggest challenges
that your team faces, right?
John Gillham (18:43):
So for sure it is for sure it's it's a, one of our
biggest concerns, the data saysotherwise right now, which is
interesting in that are when soI think you're exactly right
that when, like when judge UBTfirst came out, it was hard to
make it right differently thanit's like forced style.
Like it was like, but then whenGPT four came out, You could
(19:04):
say, write a Nobel Peace Prizeaccepted speech in the style of
Dave Chappelle.
And no one would think that hadbeen AI generated pre that
written piece of content.
And so I think humans ability totell went out the window because
the diversity of what it couldcreate and the directions that
you could provide it became sosignificant.
What we have seen from Our ownmodel, whatever it's picking up
(19:27):
on whatever our own AI ispicking up on, we've seen the
new model will come out, ouraccuracy rate would drop from,
let's say it was 98 percentwould drop down to 85%, big,
unfortunate drop, have to trainup on that new model.
And then we would close thatgap.
What we have seen with thelatest models, basically no drop
off in 4.
0, Cloud 2 to Cloud 3 wasminimal drop off, Gemini Pro, no
(19:49):
drop off.
We're seeing a lot of thesemodels are trained on same data,
common crawl of the web, samehardware, same sort of
foundational technology aroundtransformers and we're seeing
this in 4.
0.
Diminishing a gap get opened upwith every new model over our
capability of detection.
That's what we're seeing rightnow.
(20:11):
Pretty hard to bet against theexponential growth, the
exponential amounts of moneythat are being poured into the
space.
And when GPT 10 comes out, arewe going to be able to do that?
Accurately detected.
I don't know, but I'd say ourcurrent sort of our gap to new
models has been closing with ourcurrent detection capability.
And I think that's leading to usbe on the side of okay, we're
(20:32):
seeing a little bit of aplateauing right now around
model capabilities.
And a right intelligence ofthese models.
The jump from GPT two to GPTthree felt bigger than the jump
from GPT three to GPT four.
Will this will the same exists?
Well, that will we see the samesort of Perceived diminished
(20:53):
jump on the next models.
I don't know.
Jed Tabernero (20:57):
Very interesting.
So using AI to detect AI, westarted looking into kind of the
other platforms in this spacejust to understand, Hey, what's
the competition looking like?
There's other models probablydoing a little bit of, the same
industry.
What would you say?
Sets yours apart than theseother companies and a lot of the
(21:21):
cool features I want to go overhere later But I'd love to hear
from your perspective you knowWhat sets you apart from the
other model
John Gillham (21:29):
Yeah, I think it's a two things and they're
related, but I think we madepretty quickly the decision that
like the world we understoodwhere was digital marketing
content being published on theweb.
That was the world that weunderstood.
Where we had felt like an unfairadvantage.
That's who we started buildingfor.
And so we build for basicallycopy editors that exist in any
organization that are getting apiece of content and then need
(21:52):
to publish that piece ofcontent.
So copy editors, toolkit, AIenabled toolkit.
And that's what we're building.
Core product is AI detection.
Because of that, because of whowe're building for, it's a far
more B2B than a B2C play.
Our free tier is far morelimited.
Our our pricing doesn't need tobe.
We're not competing forstudents.
(22:12):
And so because of that, we cangive more horsepower to our
detection team to say, run, runharder on the compute.
We can have a false positive todetection rate that is tuned to
our users use case and notneeding to be the sort of super
general detector.
And so because of all thosethings our data set gets tuned
(22:34):
to our users as opposed to beingthis sort of general purpose
detector.
And so I think that decision ofwho we're building for has led
to a bunch of other sort oftweaks along the way with the
model that has led to repeatedlyshowing up as the most accurate
aid detector.
AI research team incrediblysmart, but we're going up
against teams that are alsoincredibly smart I think the
(22:55):
combo of sort of that this AIresearch team being incredibly
good And then a bunch ofdecisions to get them aligned
with being capable of buildingGive them the tools to build the
best detector.
So most accurate detectors isone and then for our users,
which consistently proven bystudies.
And then the second being we'repretty clear on who we're very
(23:17):
clear on who we're building for.
Shikher Bhandary (23:19):
Big kudos to the fact that y'all are the most
accurate.
AI detection tool in the marketright now.
So that is in itself incredible.
So the stakeholders right now,the customers that y'all are
actively working with hand inhand, not just copy editors,
(23:40):
like you mentioned, but also,these marketing companies, maybe
educational institutions, likehow are you thinking through the
actual Customer cohorts totarget and build those
relationships with.
John Gillham (23:53):
Yeah, so we, I think we're, I think we're the
only tool that actively liststhat we're not for academia.
Shikher Bhandary (23:59):
Okay.
John Gillham (24:00):
that I think, so we do have a lot of academia
that uses us because we are themost accurate.
The sort of amount of toolingthat you need to build around
academia to be confident it'sbeing used.
In the right way is more than wethat's not the problem.
Like it's a big problem.
It's just not the problem.
We're focused on.
And so we know academia is usingus.
We don't love it being used foracademia.
(24:21):
So we false positives do happen.
And the amount of tooling youbuild around that to deal with
that within an academic settingis different than what we build
within a for for writers wherewe can have a free chrome
extension and some other toolingthat helps deal with false
positives in the writer.
Editor relationship.
And so that's like the one sortof unique condition for us.
(24:42):
And then the rest of our usersmostly fall into the digital
marketing world.
So web publishing world,whatever we want to call it, but
getting content from a writer.
Reviewing the content,publishing on the web, and so
any company where a copy editor,where somebody functions as a
copy editor then our tool can beuseful, and that can be
incredibly small companies, oneperson operations that has one
(25:05):
writer for their one website,and it can be incredibly large
organizations that havethousands of writers and
hundreds of editors.
Really focused on that role, orpeople functioning in that role.
Shikher Bhandary (25:16):
Yeah.
And it kind of fits, it probablyfits better to what your
incredible expertise is in anyway, right?
Because you've built what is it?
Three companies within thecontent space.
So you probably know exactly theworkflows exactly where in that
workflow and those individualsmaking these decisions.
John Gillham (25:38):
For sure.
Off the start.
I've been, I would have agreedwith you wholeheartedly a year
and a half ago.
I think after enoughconversations, I've been pretty
humbled at Oh, maybe I didn'tunderstand this space.
I didn't understand all parts ofthis space as well as I thought
I had.
The use case for somebody in alarge organization will be
different than a marketingagency, which would be different
than a website.
So in general, yes.
(25:59):
But I think Yeah, still lots tolearn on exactly how everyone's
workflow.
Even though it's a similarfunction works, but it is
definitely the space that weunderstand the best.
Jed Tabernero (26:09):
is there a specific feature that you're
most excited about John?
Of what you're providing todaythat be B to C or B to B
customers that you have Not
John Gillham (26:20):
This would be a weird answer.
It's the one I'm mostdisappointed in, but I'm also
most excited for.
So we built a fact checker soheavily reg enabled.
Fact checker, super intensive interms of going out, finding the
information for every statementof fact, trying to do reg,
laying it in, overlaying it withan LLM to then provide an answer
(26:41):
on, is this factual or not?
It's not very good.
So we're really excited for it.
It's still in beta.
It provides it's an aid in termsof providing research to help
people in the process of factchecking when they get a piece
of content.
I think hallucinations andfactual accuracy of what
(27:02):
elements output.
Is a pretty massive problem.
That is hard to solve by justenriching, increasing the data
because the so the unbalancedthe creative nature of these
models really, really hard tokeep them within.
Within parameters and factualaccuracy, even with all with a
ton of constraints put on uponthem.
(27:23):
So that's the one I'm excitedfor it because I think it's
solvable eventually.
And so we're keeping a reallyclose eye on when can this added
effort that we can Inject interms of reg and understanding
the web and trusted sources.
When can that be matched up withan LLM that will provide that
(27:44):
level of fact checking that wecan look at and be 99 percent
plus confident is accurate andwe're not there yet.
Jed Tabernero (27:53):
I think I'll just share what my favorite feature
right now is.
It's the readability piece, howcleanly it comes out when you
put something in.
Which I think if you guys haveused other tools like Grammarly,
it's it's in the same light.
But I think the fact that.
Right underneath the output.
First of all, you color all ofthe, the unreadable sentences,
the long sentences.
(28:14):
It just basically told me howshit of a writer I am.
But it's really
Shikher Bhandary (28:17):
I've been telling that to you like for two
years, dude.
Jed Tabernero (28:21):
it'll tell you something's you've got over 20
syllables in this sentence,right?
Yeah, that's difficult.
That's probably difficult tocapture.
And you don't realize thesethings when you're writing
stuff.
So I can see how it was builtfor those content marketers as
well, because these are thestuff, of course, me and Chick
are having this as a passionproject, this is one of the
things that we care about themost.
So very interesting.
(28:42):
You also have a feature onparaphrasing, which I think was
difficult for me to understand,but that's trained on something,
right?
Trained on a tool that, thatdoes the paraphrasing for you.
John Gillham (28:54):
Yeah.
This, what, one of the, one ofthe most fun aspects of this
role has been the cat and mousegame.
And just the, like the constantbattles of like launch detector.
And I was too dumb to listen tothe feedback that we got, but it
was like the feedback we gotfrom day one was is there a
button that I can just pressthat will make it past the
(29:15):
detection?
And I'm like, yeah, No, we can'tbuild that like our tool would
be useless with that which iswhich is true.
But I got that feed that requestso many times that I was like,
well, why didn't I see coming?
What came next, which is nowthere's all of these tools that
attempt to bypass us.
And so one of the most commonways, especially early on to buy
(29:36):
and still works for a lot ofother detectors was to use a
paraphrasing tool.
So using and that produces a newpattern.
And so you'll it'll You cancreate content using an LLM and
then remove, I won't watermarks,not the right term because that
has a whole other meaning to it,but remove the pattern
recognition that comes byparaphrasing.
(29:56):
And so that's, that was one ofthe earliest methods of trying
to bypass detection.
There's been a bunch more since,but there, yeah, anyway it's fun
to, we have a red team and ablue team.
Red team's always trying to beatour detector, find out what
tools are available to beat ourdetector and then build a data
set to try and learn against it.
Jed Tabernero (30:12):
I love that, that that's part of the culture.
There's a red team and a blueteam.
John Gillham (30:17):
Yeah.
Yeah, it's it's it's unique andfun for sure.
Jed Tabernero (30:20):
no, it makes it a lot more fun when I saw these
tools, just.
Thinking about somebody likemyself, who's in this space
where we're writing a lot ofstuff, we're talking about a lot
of things and then transcribing,then writing a lot of writing.
I thought to myself, this wouldbe really dope to integrate with
some of the native tools that wealready have, right?
Some of the workflows that wemight have in our space to just
(30:42):
see, Oh, you know what, let'sjust, when we write something,
make sure it goes through thistest to ensure that it's
readable, so that my shitwriting doesn't get published.
But, things like that.
A lot of people.
Are now publishing onlinecontent, as we said, and using
AI to do so.
So it's just like really useful.
Just question for you, anyintegrations in the future,
anybody courting you to become,part of their tool, et cetera
(31:05):
would love to hear about anyfuture plans like that.
John Gillham (31:09):
Yeah.
So we have a chrome extension.
That works that is tightlyintegrated into Google document
workflow.
Not so much from a creationstandpoint.
And so for some, to some extent,like what grammar really is to
writers, we are aiming to be toeditors.
And so if there's integrationsthat make sense on under that
lens, then yes, I don't think wewill aim to be.
(31:30):
Writing aid.
And so I think the level ofintegrations that occur if we
were focused on being a writingaid increases with a current
focus not opposed tointegrations, but they got to
make sense from that use case ofa copy editor.
And in those cases like we havea Chrome extension that is very
helpful for people to understandthe originality authenticity of
(31:52):
a Google document.
Shikher Bhandary (31:55):
That's great.
The way the media or the wayeven the folks in the industry
compare models is through, abunch of these.
technical jargon, right?
Tokens, parameters.
Oh, this is this size.
This is this size.
So specifically on your product,does it matter how advanced or
(32:17):
how many tokens or parameters ithas to be able to detect
something that was generated bya model that does have those
extraordinary numbers of tokensor parameters?
Is it something that you'rethinking through?
John Gillham (32:31):
Similar that sort of earlier conversation around,
like we're seeing this sort ofdiminishing,
Shikher Bhandary (32:37):
Okay.
John Gillham (32:37):
gap that is showing up with every new model
that comes out.
You know, exponential growth ofparameters.
Output is.
Better in some ways.
But then our capability ofdetecting it doesn't drop off.
And I'm hesitant to like projectout in this world.
What, what's coming, but I cansay what's happened, his history
being the last two years.
(32:58):
What we've seen is that thissort of exponential growth Of
parameters, model sizes,training costs data consumption
has not led to to the abilityfor these LLMs to create content
that whatever our models, ourdetectors, most accurate, but
there's other detectors that aredecently accurate.
Whatever they're these detectorsare capable of picking up is
(33:20):
staying is staying in placeright now.
And so we're seeing, I'd saywhere it's been interesting and
the biggest sort of gap that hasexposed.
There's all these differentcriterias for these.
for these models.
Some of the open source modelshave been, even though they
might have a smaller parametercount, smaller training size
they have produced someinteresting variability in, in
(33:43):
accuracy.
So like Mistral in particulardid, again, easy enough to
close, but that was one of oureven though most open source
models up to that point had notbeen I challenged I've been
quite easy to detect.
Shikher Bhandary (33:55):
It threw a wrench into things.
Yeah.
John Gillham (33:57):
yeah.
Yeah.
So that was interesting.
Shikher Bhandary (33:59):
Got it.
Yeah.
I was just thinking because thetalk of the town is LLMs, SLMs,
action models, and things likethat.
And I was just wondering wherein that whole category does an
AI detection for those LLMsactually sit.
But it's great that now thatyou're up 4.
0 or 4.
0 from here, it still isaccurate on a high percentage
(34:25):
basis.
John Gillham (34:26):
Yes, we're the most accurate we have ever been
on the latest model.
Our gap has been closing.
This could aged extremelypoorly, right?
GPT five could come out and makedetection totally impossible.
Oh, yeah, we'll see.
Shikher Bhandary (34:39):
you're doing your best.
You're doing your best.
Jed Tabernero (34:42):
Yeah.
Can you imagine if GPT 5 cameout and that's its entire goal
was to consume AI detectionsoftware?
Oh my god.
John Gillham (34:51):
You know, I just listened to Mira the CTO at
OpenAI talking about detectionand I, I think I understand why
they're detector.
So they had launched their owndetector, but they were given a
certain, like it was had to befree, had to be super low false
positive.
The result was pretty muchuseless on detection.
And so I understand and viewedwith a lens of certainty.
So I understood why.
(35:11):
Given the criteria that wouldhave been provided to their team
to build a detector, they wouldhave had to fail.
And I think they don't want, Ithink they would want their
content to continue to becapable of being detected
because It helps to decrease thesocietal harm that their product
will be capable of producing.
And that's at least the wordsthat they're saying.
(35:32):
They would love to havewatermarking.
I think watermarking will neverbe a solution within text.
But yeah, so that's what I wouldsay is it's interesting to yeah
be trying to read the tea leavesof what OpenAI truly cares about
moving forward.
Jed Tabernero (35:49):
Interesting point on the societal risk.
We touched on it in thebeginning, right?
To say that, hey, what's thescenario this actually might be
hurtful.
And we looked into someexamples, one of which was like
a book on Amazon.
That had very questionableinformation about foraging
mushrooms is it was, that wasyour bullet, right?
(36:10):
Chikor,
Shikher Bhandary (36:11):
Yeah.
Yeah.
And it was actually a link thatJohn shared and baby formula
stuff was also there where nowpeople are questioning.
Jed Tabernero (36:20):
because you, you don't think about these right as
a risk.
And I'll tell you why thisspecific example was so funny to
me in the beginning was becausewhen ChadCPT first came out, I
messaged Chikor and I said,look, I found another way to
make money.
I'm going to make a book and I'mgoing to have chat GPT write me
every chapter, to make thisbook.
(36:40):
And it was just interesting tome that the example that was
provided was a
Shikher Bhandary (36:44):
Literally someone who thought about that.
Jed Tabernero (36:46):
AI.
Yeah.
Literally someone who thoughtabout that and then put it on
there and sold, probably made alittle bit of money out of it.
People don't think about thatsocietal cost when it comes to
these like super useful thingsfor humans.
So the way that I think aboutoriginality AI in general are
guardrails.
To these, really crazyinnovations.
That's why I think it's sointeresting that we covered just
(37:08):
a company before thishighlighting, the power of what
AI can do and AI agents.
We talked a lot about AI agents,and we've been talking about a
lot of the positives ofartificial intelligence.
And I.
I appreciate that for thisepisode, we're actually able to
step back and say, look, this isone thing what we're doing with
an ad using AI actually toreduce societal, harm, which is
(37:31):
pretty awesome.
Have people who are talking toyou about originally AI over
index on the piece that you arereducing societal harm.
John Gillham (37:38):
I think there is a societal importance to the
company that, that I didn'tinitially set out to, to build,
and it is being used in waysthat I hope is reducing harm.
Again, a lot of our focus is onthe, that were the, the specific
vertical we're in, and in, inthat vertical, I think there's a
significant misunderstanding inthat space, where right now we
have writers, You need to useour platform to make sure their
(38:01):
content passes AI.
This is a problem that theynever experienced before.
They aren't using AI.
We have false positives.
And there's this sort of harmthat comes from that because
there's writers that aren'tgetting paid because there's a
false positive.
What I think is being missed isthat short of having some AI
Detection in that workflow thatthe volume of writers goes up
(38:25):
to, the, basically the worldpopulation that's connected to
the internet for a good chunk ofwriters.
ChadGBT writes better than I do.
Sounds like better than you,better than yourself.
But
Jed Tabernero (38:34):
Just me specifically.
John Gillham (38:37):
yeah.
So if I was, if I made mylivelihood as a writer AI
detection, although theoccasional false So that can be
harmful.
It's a, it's at least defendingthat industry from being totally
wiped out by by chat GPT andother LLMs.
Shikher Bhandary (38:54):
There's a lens or there's a view of, Hey, if
this is good for the consumer,why should they care?
And this is why they shouldactually care.
Right.
Because people are just going tobe like, Hey, Content is going
to be ingested regardless,whether it's from actual human
or AI, if it's good content.
What's the harm, right?
And there's actually a harm herebecause it just leads to other
(39:16):
things.
John Gillham (39:17):
Yeah, exactly.
Shikher Bhandary (39:18):
Just wrapping up, John, this was fantastic.
When we have such guests on suchfounders, academics on, we give
them the stage to give a shoutout.
Maybe it's for a team, maybeit's for a new product release.
Maybe it's fundraising too,because we have a lot of
founders that have actually Usedour platform to connect with the
right VCs and stuff.
(39:38):
We'd love to give you the stage.
John Gillham (39:40):
Yeah, no sounds good.
But yeah, so I'd say if anyoneis working in an organization
that, that has peoplefunctioning as a copy editor and
is trying to wrestle with thesequestions that, that we wrestled
with today on what is allowable,not allowable, and whether or
not the risks are, Adequatelymanaged in your organization
with the use of a I inspecifically for us within the
(40:00):
world of content marketing.
We're happy to chat and happy tohelp people think through the
appropriate uses of it and theappropriate sort of use of
originality for mitigating thoserisks.
Jed Tabernero (40:12):
Awesome.
And do people still get Freetokens.
If you get the Chrome extension.
John Gillham (40:17):
Yeah.
So if you sign up and and get,yeah, 50, 50 free credits, so we
have the free tool that you canuse which is super limited, and
then you can get 50 free creditsto the premium tool when you
confirm your, sign up, know withthe Chrome extension.
Jed Tabernero (40:29):
Sweet.
Well, this was super awesome,John.
We learned a lot, even justdoing the research, honestly,
but talking to you was a lotbetter.
Really appreciate your time andthanks for coming on.
Things have changed, man.
John Gillham (40:39):
Yeah, thanks for having me.
Fun fun conversation.
Thanks for tuning into today'sepisode of things have changed
podcast.
We hope you found our discussionwith John Gill, him
enlightening, and that itsparked new thoughts about the
digital content we interact withdaily.
Remember.
In a world where technologycontinually evolves, staying
(41:00):
informed is key to navigatingthe complexities of digital
authenticity.
And that's what urgent Lottie AIcan help you with.
Cheers.
And as always stay curious.
The views and opinions expressedin this podcast are those of the
guests and do not necessarilyreflect the official policy or
position of things have changedpodcasts or its affiliates.
(41:23):
The content provided is forinformational purposes only and
should not be taken asprofessional advice.
Listener discretion is advised.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Las Culturistas with Matt Rogers and Bowen Yang
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.