June 16, 2025 • 23 mins
Analyst Talk - Open Secrets - Don’t Just Trust AI, Always Verify Episode: 00271 Released on June 16, 2025 Description: In this third installment of the Open Secrets series on Analyst Talk with Jason Elder, expert Jan Mondale discusses the evolving intersection of open-source intelligence (OSINT) and artificial intelligence. From personal experiences using AI tools like ChatGPT for medical research to broader implications for law enforcement analysts, Jan and Jason explore the power and pitfalls of integrating large language models into investigative workflows. Jan emphasizes the timeless importance of critical thinking, source validation, and ethical use—whether you're verifying a suspect's address or navigating the risks of misinformation. They also touch on AI's role in cybersecurity, translation, threat detection, and even the risks posed by overly trusting social media.
Key topics include:
- Verifying AI-generated information
- How OSINT and AI compare in investigative reliability
- AI's utility in law enforcement and cyber threat analysis
- Why analysts must remain smarter than the tools they use
Whether you're AI-curious or a seasoned OSINT pro, this episode challenges you to think deeper about the tools at your disposal—and how to use them wisely. [Note: Description produced by ChatGPT.] Related Links: Jan’s Notes: https://mcdn.podbean.com/mf/web/n6y2qvnkk64sebvt/OpenSecretsOSINTandAIJansNotes.pdf https://static.semrush.com/blog/uploads/media/ea/34/ea34afd0390da60dc82700c472f7d527/YIbvIN7lhZP6JJGSQvPgPmD5rSJ5H85yTxW30vNlkkwh952w5VjeLOfFIk2dVpPewO6fGCV5HN05reZF3jNikXjVpepl6xxHHE_RSRRXgnkqNp9bw-cNRn0ztDSy9XSw_a3GY-dIWlkUjpUYRBk8WZo.webp https://ignitevisibility.com/how-is-bing-seo-different-than-google-seo/ Association(s) Mentioned: Vendor(s) Mentioned: Contact: https://www.linkedin.com/in/janmondale/ Transcript: https://mcdn.podbean.com/mf/web/8mrsbshuzfvcnmmx/OpenSecrets03_transcript.pdf Podcast Writer: Podcast Researcher: Theme Song: Written and Recorded by The Rough & Tumble. Find more of their music at www.theroughandtumble.com. Logo: Designed by Kyle McMullen. Please visit www.moderntype.com for any printable business forms and planners. Podcast Email: leapodcasts@gmail.com Podcast Webpage: www.leapodcasts.com Podcast Twitter: @leapodcasts
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
Welcome to Analyst Talk with Jason Elder.
(00:01):
It's like coffee with an analyst,or it could be whiskey with an
analyst reading a spreadsheet,linking crime events, identifying a
series, and getting the latest scoopon association news and training.
So please don't beat that analystand join us as we define the law
enforcement analysis profession.
One episode at time.
How we doing?
Alice?
Jason Elder here with another LE.
A podcast.
Deep Dive, open Secrets with Jan Mondale.
(00:25):
Jan, how we doing?
I'm doing really well.
Thank you, Jason.
All right.
We're here talking about openintelligence, osint related topics as
it relates to law enforcement analysis.
Jan, what are we talking about today?
Well, I recently had an experiencewhere I had to use chat GPTA
(00:46):
lot to try to understand.
Medical information, and I realizethat that's not necessarily law
enforcement related, but it made mestart thinking about how I use Osint.
I consider myself an osintperson, but I know that , in
the back of my mind, I wonder.
How are people using large languagemodels such as Chat, GPT and other
(01:10):
platforms to do their analysis?
Are they using ai?
Are they relying on ai?
And as with osint, which has beenaround for a long time, but people
think it's new, AI has been around fora long time and people think it's new.
I have books because I,I love, I love books.
I collect a lot of books.
I have books going back to the1970s from Stanford University.
(01:32):
Talking about artificial intelligence.
One of the classes I took when I wasgetting my degree in library science
was called Knowledge Representation.
And it was essentially trying tofigure out how to teach computers
to understand natural language andwhat kind of terminology we could use
that, computers would understand, andnow of course they figured it out.
(01:54):
We can teach computers tounderstand natural language.
It's, it's quite amazing.
That part of AI is new.
It's controversial but it's very useful.
It can be very useful.
If you know how to use it carefully,as with Osint, you need to learn how to
use osint carefully so that you aren'tinvading somebody's privacy or using it
(02:14):
without doing critical thinking analysisskills on it to make sure that you
aren't being spoofed by somebody who's.
Putting something online that mightbe misinformation or disinformation.
So you have to worry about that with AItoo, because the, the language models
that teach these G chat, GPT and, andothers are based on what's fed into them.
(02:36):
Mm-hmm.
So, garbage in, garbage out, and youhave to be aware of that now, for the
most part, if you ask the questions.
Right.
I, I, because I'm a librarian I liketo know what the references are.
So when I ask chat GPTA question, Iask it to provide where it found the
information so I can double check tomake sure that it's a valid source.
The same as I would do if I werejust doing osint research on my own.
(02:59):
I would verify it.
So that's one of the thingsI wanted to talk today.
Yeah, and I think that's the bigdifference between osint and ai.
When people are doing research.
And they come across an article andthey are consuming that article.
They can see what the source is,how maybe how reliable the source
(03:25):
is, or have an idea of what thesource even is, dates, mm-hmm.
Other information to helpcorroborate that information.
I feel with ai, I don't think people do.
, What you're suggesting there, whichwould be to ask about the resources.
I think people ask, ask it a question,and then it gives them an answer
(03:50):
they aren't even, I. Consideringor may not be considering the,
the source of that information.
Exactly.
That's a big problem.
It's a big problem.
If you're reading a news article from say,USA today, and it's citing another article
that is the basis for that article.
I always check, I I click that link.
(04:10):
If they don't provide a link,I don't believe it at all.
I just, I, what I do is I copy parts of.
What the article might sayand run a search on that.
Mm-hmm.
To see if I can find thesource of the original source.
Almost all open source intelligenceis secondhand information.
It's not original source.
And so to get to the original source,you, you have to trust that they
(04:32):
put the links in there and you asan analyst need to make sure that
you check those links, for example.
I mean, it can be something as simple as.
A suspect gives a law enforcementofficer an address and that address.
In some states now we're getting betterabout making sure that people's driver's
licenses are accurate and authoritative.
(04:53):
But there are, there are some states, andWashington State used to be one of them
that didn't necessarily do a very goodjob of checking to make sure that the
address that it applicant for a driver'slicense gave them was a real address.
That doesn't happen.
Washington State changed the law.
So now we have to have valid addressesbefore we can get a driver's license.
(05:15):
But when I was an analyst a lawenforcement analyst, I remember
times when I, a law enforcementreport would, and it's.
It's kind of not really opensource information, but you
could find that person's addressby doing a Google map search.
And I couldn't believe , how oftena really bad person, not your
average person, has a real addresson their license driver's license.
(05:39):
But how often bad people use a postoffice box like a mail post or a UPS
store storefront where you can rent a box.
Not
necessarily the US Postal Service.
'Cause they do have rules about youcan't put PO boxes, but you don't.
When you put my, my businessaddress is a mail post address.
I don't give it out very often.
(06:00):
I just use it because it's easierto get my business mail there
than it is at my residence.
So I don't have it on my driver'slicense or anything, but I could.
I could have, in the old days, backbefore Washington changed their laws.
I could have given that as my.
Address for my driver's license.
And, and nobody would've beenthe wiser, the police officer
(06:20):
wouldn't have been the wiser.
The analyst trying to track down thatcriminal wouldn't have been the wiser
unless they did an open source Googlemap search and discovered, oh, that's
a mail post in Renton, Washington.
Not, not.
Residence.
So those are the kinds of thingsthat , people need to verify.
And with ai, , I always ask whenI'm using AI for something specific,
(06:45):
like a question that I'm asking, Ialways ask it to give me the sources.
I think that's one of the things thatI think people in law enforcement
especially that have never usedit, the question becomes, well,
how do you know that it's right?
What it's telling you is accurate?,
It the same way if the same wayyou check any kind of intelligence
you get, you verify it.
(07:05):
You
put on your
critical thinking hat and you verify it.
Yeah.
I think that also gets back on whatyou're asking it to do and 'cause I, I
see it as from my perspective when I'musing chat, GPT or another AI software
is, I am usually asking it to help mewith something maybe technical right.
(07:26):
SQL code or an Excel formulaor something to that.
Effect Right.
Obviously I am then taking whateverthey're giving me and testing it out.
And that's the verificationprocess, right?
With that.
Right.
Exactly.
Whereas
if you're getting something where it'slike, tell me what the crime stats,
(07:47):
in Santa Monica are, then you'regonna have to go through some kind of
validation process to know, get intothe source and be able to verify.
What it's reporting back asthe crime stat to Santa Monica.
Right.
The, the thing is though, theAI is a, can be a great tool.
Mm-hmm.
Especially for analysisof large amounts of data.
(08:09):
It's, it's just, you can feed it awhole bunch of information and say.
Give me a summary of this.
And or you can nowadays because,and it's only been a couple of
very short years since mm-hmm.
It became so ubiquitous.
But nowadays when you do a Googlesearch or a Bing search, you're
automatically getting AI results.
(08:31):
Mm-hmm.
Part of my, if you put this up in, inthe show notes I included a, a graphic
that I found online, and I'm sorry Ididn't put a, a source on here for this.
I think I have it somewhere.
I'll have to put it on in.
I'll send it to you in an email.
Mm-hmm.
So.
So I can do what I should havedone and include the source.
But anyway, this graphic shows theranking factors that Google and being
(08:53):
use it's right under, underneath being,it says, uses open AI technology, which
is chat GPT Technology to understandcontext and deliver on user intent.
So right there in the very first line ofthe being ranking factors for key words,
it uses open AI technology to understandthe intent of the search question.
(09:15):
Mm-hmm.
So being admits it right up front.
We use ai.
So ai, machine learning, all ofthese automation tools, they're able
to scan massive amounts of data.
They can identify connections.
For example, I, I don't really recommendit because it's not very thorough yet.
When you're doing a peoplesearch, say you're looking for
(09:37):
Jan Mondale online, you can type.
My name into chat, BT, and it givesyou a pretty good idea of who I am,
but that's me being purposely openon, on the internet because I like
to, when I do test searches, I searchagainst myself to see what I can
see, what kind of search results Iget for different kinds of queries.
Mm-hmm.
(09:57):
And so I have tested chat GPT against myown name and it gives a pretty decent.
Result.
Most of it's from my LinkedIn account.
You can tell just by looking at it.
Someone I know recently asked me for helpfinding out, verifying whether somebody
was, had a military, was a veteran,and that's what their question was.
(10:18):
And I said, oh, that's really hard.
Unless you're the person, you aren't gonnaget it from a government source unless
you are the veteran and you're trying toget information about your own record.
It's, it's pretty close.
Hold information on whether or notwhat your military records are.
I said you could try social mediasearching, but I said you are relying, you
still have to figure out a way to verifyit because you're relying on that person.
(10:40):
For example, me, if I put onmy LinkedIn account that I am
a veteran of the Air Force.
You are relying on me telling thetruth on my social media account.
Mm-hmm.
So there is that, that worrythat somebody's putting false
information up there and chat.
GPT is relying on that false informationto give you your infor your answer.
(11:00):
So you, again, you have to verify, youhave to figure out a way to verify it.
You can have a good hypothesisthat I was in the Air Force based
on my LinkedIn account, but youcan't verify that very easily.
Because unless you can figure outa way, if I were dead, maybe you
could find whether through find agrave if I'm buried in a veteran's
(11:20):
cemetery or something like that.
I mean, it's, it's just notan easy, an easy thing to do.
Especially if that person, I, I onceon social media, open source, I. Public
social media page, a person posteda picture, and I totally forgot that
that picture, just because he posted iton that day, doesn't mean that that's
(11:42):
the day that that picture was taken.
Hmm.
And so I used that as confirmation thatthat person was where he said he was.
And that was stupid.
That was very stupid.
That person did that on purpose.
Because he wasn't wherehe was supposed to be.
And so he totally fooled me, didn't fool.
Fortunately I was working with the FBIon that case, and the FBI wasn't fooled.
(12:05):
So because they had other sources,they had flight, flight data that
showed that he was on a flight whichis not open source information.
So sometimes, especially lawenforcement analysts are, are kind
of fortunate because they can relyon governmental sources that aren't.
Publicly available information toverify what might be on social media.
(12:26):
And so with ai you always have to becareful of that, but it is wonderful for
correlating and giving a comprehensive.
View, quick view ofwhat might be available.
It's also really great.
I know that we have a lot of translationcapabilities with Google search and
Google Translate and being translateopen AI can do it really fast too.
(12:50):
Just upload your text yourforeign language text.
Into an open I ai search box andask it to translate it to English.
And it's really fast.
And it'll even summarize it for you.
If you ask it, ask it to, like ifit's a foreign language journal
article or something like that.
So there are reallygood reasons to use it.
It's really, really helpfulfor threat analysis.
(13:12):
I am, I'm a cyber threat analystnow, and so it's very helpful.
I don't use it 'cause my companydoesn't allow us to use ai.
But.
If, but other companies do, thereare large companies that like that,
that people hire just to do their,to help them with their large amounts
of data, and I know that they use itsilo or authenticate, recorded future.
(13:37):
What are some of the others anyway?
Big companies that are private orcommercial intelligence companies use
open AI or, or their own, 'cause a lot ofthem have built their own large language
models and they feed the data in and they,they use that data to provide support
to the companies that, that, that arehired by them or they, hi, hire them.
(14:00):
Mm-hmm.
So some companies don't.
Allow their own analysts touse it, but they rely on other
companies that are using it.
However, those companies, I do haveto say, they're really good about
identifying what they've used AI for andwhat they've used a human analyst for.
So they're really good about that.
Yeah.
So.
So, yeah, it's definitely used.
(14:22):
It's not used by everybody.
I highly recommend if your, ifyour organization, if your law
enforcement organization allowsyou to use it, that you check it
out, see what you can find out.
It's great for multi-language stuff.
It's great for quickly finding instancesof a username across different platforms.
That's one of my, my techniquesfor finding people is a lot of
(14:44):
times people use their usernameon more than one platform.
Mm-hmm.
And so you can get pretty good results ifyou type a username into chat GPT and say,
can you find where this username shows up?
On the internet or somethinglike that, and it'll do a pretty
good job of finding all of thedifferent sites or IP addresses.
It's good for doing large scaleanalysis of tactics, techniques, and
(15:08):
procedures that cyber criminals use.
To attack organizations so you canset it up to autogenerate Alerts for
Threat Actor Activity for Spam thatmight be coming in to your organization.
Phishing detection.
Detecting questionable emailsthat are sent in order to try
(15:30):
and get you to give information.
It's, it's a very common practicefor cyber criminals to send phishing
emails to organizations claiming tobe that organization's security team.
And they want you to, they wantyou to change your password.
So, I mean, most it people are smartenough not to fall for that, but there.
(15:51):
They catch every, they catch people.
Every now and then, somebodyaccidentally goes, oh yeah, I guess
I do need to change my password.
And they put in their old passwordand their new password, and all
of a sudden a cyber criminal hasac access to their organization.
So you, you just, it's really good for.
For things like that to help youwith large amounts of information.
(16:11):
Right, and I've already pretty muchgone over the, the disadvantages.
The fact that it could be, it, the, theAI could have been fed false information.
And it's gonna give you falseinformation if it's not given good
information, and it can be biased.
Mm-hmm.
That's a, a complaint a lot of timesfrom different sides of the political
spectrum is that AI is biased by, by theimplicit bias biases of the person who's.
(16:40):
Something that annoys me as a, asa former librarian is a article I
read last week about, about Googleingesting huge amounts of copyrighted.
Books into their AI model andknowing that they will get sued for
it, but knowing that they're sucha huge company that they'll win.
Hmm.
And even if they have to pay,pay something, it doesn't matter.
(17:02):
They're such a huge companythat they're super rich and
that's what this article said.
And that, that just, you,sometimes you get bad information.
That's not necessarily wrong.
Information just shouldn't be there.
So that kind of thing.
It's, it has its disadvantages.
A lot of people worry that it's, ohno, AI is gonna take over the world.
It'll only take overthe world if we let it.
(17:24):
If, if you feed that information intoit, if you allow it to be your only
source of information, then yeah, it'sgonna probably take over your system
and, and you're not gonna get a verygood reputation for your analytical
skills if you use it to the exclusion of.
Of your own really smart, analyticalskills that no matter what,
(17:44):
you're smarter than a computer.
Because computers only areas smart as you make 'em.
One of the things that , , I uploadedwhen I sent you my show notes was
a little graphic and I just wantedto point out that if, if you do
post this in the show notes that I.
I, I fed my notes into chat GPTand asked it to make a graphic
for me, and that's what it made.
(18:05):
So you can use it forthings like that too.
Yeah.
Yeah.
And it, I've, I've used itfor that as well, and so it's
interesting how some little thingsthat'll the, it'll get wrong.
Either misspellings or the graphicwill be a little odd looking or
it's funny how it's not quite right.
Made
the, made the little computer, thelaptop computer a little too big.
(18:27):
Yeah,
yeah.
To cover covering up some of the text.
Yeah.
But anyway, that's just what, yeah.
We'll put that in.
The show notes mean outta my notes.
Yeah.
We'll put that in the show notes.
I, okay.
I, I think for, for law enforcementand for analysts, I, I think
it's really difficult to.
Take AI seriously interm in terms of osint.
(18:48):
And I think what I mean by thatis , you're only scratching
the surface when using it.
I, I would only envision using itwhere I, I'm just curious what osint,
I mean, AI on Osint on a particulartopic would kick out knowing that
I'm already probably well versed inwhat I should expect to see and Yeah.
(19:10):
But it could.
Possibly think, take you down a paththat you didn't expect, or maybe it's get
gets you a lead that you, didn't expect.
So I would think that'spretty superficial.
Like you're not going much moredown the rabbit hole than that if
you're using AI for, for osint.
Yeah, I don't think, I mean,it kind of just depends on
(19:32):
how much you wanna trust it.
I'm trying to find on my,unfortunately it's really weird.
Oh, here we go.
I did, I did a search, this was
a couple of weeks ago, not, not knowingthat this was what I was going to
do the, the podcast about, but I, I
did it.
I did a search.
And on who is Jan Mondale And accordingto Chachi pt, it, it searched two sites.
(19:57):
And one of the sites wasyour site Leap podcast.com.
Oh, the other one was, the other one
was LinkedIn, so I thoughtthat was really interesting.
Yeah.
But according to this, I am.
A cyber threat
analyst with a diverse background,including service in the military
and working in Ephesian Center.
I, I was recruited by the CIA aftermy retirement from the Air Force.
(20:19):
It doesn't say that leveragingher expertise in library science
and intelligence analysis,I mean, it, it gets a lot.
It doesn't get this very well.
Mondale, it doesn't say Jan Mondaleor, or Ms. Mondale or anything.
It just says Mondale is currentlyworking for a major retailer.
Her career is highlightedby her inquisitive nature
and extensive knowledge.
(20:39):
She even contributed to thwartinga potential terrorist attack.
I think they might have gottenthat from your show notes.
'Cause I don't think Ihave that on LinkedIn.
Additionally, she has personal interest in
motorcycles and is a memberof the Iron Bud Association.
Yeah.
That you, we talked about.
I remember that.
Yeah.
So I, I think, I think other than LinkedInthat your show notes are probably it's
(21:01):
primary, but I thought that was funnythat it only searched two sites and.
Yours was one of them.
Nice, nice.
Yeah.
Yeah.
As we wrap up , there's certainly , you'regonna have more and more talk about ai.
It's going to be a populartopic for decades to come and
certainly have gone through today.
(21:22):
Some of the.
Pitfalls and uses for, for ai In terms ofosis, just, I want just wanna give you the
last word on any advice or any, anythingelse that you want to add to the topic.
As with all intelligenceanalysis be a critical thinker.
I mean, as you can tell from whatI read about myself just now,
(21:42):
it's not necessarily always right.
Mm-hmm.
And so.
I, I just, I think you reallyhave to verify, verify, verify.
There you go.
If, if all Intel analysts would verifywhat they, what they find and not just
automatically go like I did with theguy who intentionally posted wrong
(22:03):
information on his Facebook page in orderto convince whoever might be looking that
he was where he was supposed to be whenhe wasn't where he was supposed to be.
You just always haveto think the bad guys.
Are using AI too.
Mm-hmm.
And so you've gotta be as smart as, asthey think they are, and you have to be
smarter than they are so that you aren't,you aren't falling for bad information.
(22:26):
But using it for good becausethere is really, I, I think that.
In some ways my husband's life mighthave been saved because of chat GBT last
month because I questioned everything thedoctor said and I, I really think that
that is something it can be used for.
For great.
Good.
You also have to verify.
(22:47):
All right, Jan, thank you again.
If, for the listeners, if youwant to email the show, email
us at lea podcasts@gmail.com.
You can also find us on Facebook,LinkedIn, Twitter and many other outlets.
If you have any questionsor concerns let us know.
, Until next time Jan, thank you forthe open secret and you be safe.
(23:11):
Thank you, Jason.
You too.
Talk
to you next time.
Thank
you for making it to the end of anotherepisode of Analyst Talk with Jason Elder.
You can show your support by sharingthis and other episodes found
on our website at www podcast.
Dot com.
If you have a topic you would likeus to cover or have a suggestion for
our next guest, please send us anemail at Elliot podcast@gmail.com.
(23:34):
Till next time, analysts keep talking.
Welcome to Analyst Talk with Jason Elder.
(00:01):
It's like coffee with an analyst,or it could be whiskey with an
analyst reading a spreadsheet,linking crime events, identifying a
series, and getting the latest scoopon association news and training.
So please don't beat that analystand join us as we define the law
enforcement analysis profession.
One episode at time.
How we doing?
Alice?
Jason Elder here with another LE.
A podcast.
Deep Dive, open Secrets with Jan Mondale.
(00:25):
Jan, how we doing?
I'm doing really well.
Thank you, Jason.
All right.
We're here talking about openintelligence, osint related topics as
it relates to law enforcement analysis.
Jan, what are we talking about today?
Well, I recently had an experiencewhere I had to use chat GPTA
(00:46):
lot to try to understand.
Medical information, and I realizethat that's not necessarily law
enforcement related, but it made mestart thinking about how I use Osint.
I consider myself an osintperson, but I know that , in
the back of my mind, I wonder.
How are people using large languagemodels such as Chat, GPT and other
(01:10):
platforms to do their analysis?
Are they using ai?
Are they relying on ai?
And as with osint, which has beenaround for a long time, but people
think it's new, AI has been around fora long time and people think it's new.
I have books because I,I love, I love books.
I collect a lot of books.
I have books going back to the1970s from Stanford University.
(01:32):
Talking about artificial intelligence.
One of the classes I took when I wasgetting my degree in library science
was called Knowledge Representation.
And it was essentially trying tofigure out how to teach computers
to understand natural language andwhat kind of terminology we could use
that, computers would understand, andnow of course they figured it out.
(01:54):
We can teach computers tounderstand natural language.
It's, it's quite amazing.
That part of AI is new.
It's controversial but it's very useful.
It can be very useful.
If you know how to use it carefully,as with Osint, you need to learn how to
use osint carefully so that you aren'tinvading somebody's privacy or using it
(02:14):
without doing critical thinking analysisskills on it to make sure that you
aren't being spoofed by somebody who's.
Putting something online that mightbe misinformation or disinformation.
So you have to worry about that with AItoo, because the, the language models
that teach these G chat, GPT and, andothers are based on what's fed into them.
(02:36):
Mm-hmm.
So, garbage in, garbage out, and youhave to be aware of that now, for the
most part, if you ask the questions.
Right.
I, I, because I'm a librarian I liketo know what the references are.
So when I ask chat GPTA question, Iask it to provide where it found the
information so I can double check tomake sure that it's a valid source.
The same as I would do if I werejust doing osint research on my own.
(02:59):
I would verify it.
So that's one of the thingsI wanted to talk today.
Yeah, and I think that's the bigdifference between osint and ai.
When people are doing research.
And they come across an article andthey are consuming that article.
They can see what the source is,how maybe how reliable the source
(03:25):
is, or have an idea of what thesource even is, dates, mm-hmm.
Other information to helpcorroborate that information.
I feel with ai, I don't think people do.
, What you're suggesting there, whichwould be to ask about the resources.
I think people ask, ask it a question,and then it gives them an answer
(03:50):
they aren't even, I. Consideringor may not be considering the,
the source of that information.
Exactly.
That's a big problem.
It's a big problem.
If you're reading a news article from say,USA today, and it's citing another article
that is the basis for that article.
I always check, I I click that link.
(04:10):
If they don't provide a link,I don't believe it at all.
I just, I, what I do is I copy parts of.
What the article might sayand run a search on that.
Mm-hmm.
To see if I can find thesource of the original source.
Almost all open source intelligenceis secondhand information.
It's not original source.
And so to get to the original source,you, you have to trust that they
(04:32):
put the links in there and you asan analyst need to make sure that
you check those links, for example.
I mean, it can be something as simple as.
A suspect gives a law enforcementofficer an address and that address.
In some states now we're getting betterabout making sure that people's driver's
licenses are accurate and authoritative.
(04:53):
But there are, there are some states, andWashington State used to be one of them
that didn't necessarily do a very goodjob of checking to make sure that the
address that it applicant for a driver'slicense gave them was a real address.
That doesn't happen.
Washington State changed the law.
So now we have to have valid addressesbefore we can get a driver's license.
(05:15):
But when I was an analyst a lawenforcement analyst, I remember
times when I, a law enforcementreport would, and it's.
It's kind of not really opensource information, but you
could find that person's addressby doing a Google map search.
And I couldn't believe , how oftena really bad person, not your
average person, has a real addresson their license driver's license.
(05:39):
But how often bad people use a postoffice box like a mail post or a UPS
store storefront where you can rent a box.
Not
necessarily the US Postal Service.
'Cause they do have rules about youcan't put PO boxes, but you don't.
When you put my, my businessaddress is a mail post address.
I don't give it out very often.
(06:00):
I just use it because it's easierto get my business mail there
than it is at my residence.
So I don't have it on my driver'slicense or anything, but I could.
I could have, in the old days, backbefore Washington changed their laws.
I could have given that as my.
Address for my driver's license.
And, and nobody would've beenthe wiser, the police officer
(06:20):
wouldn't have been the wiser.
The analyst trying to track down thatcriminal wouldn't have been the wiser
unless they did an open source Googlemap search and discovered, oh, that's
a mail post in Renton, Washington.
Not, not.
Residence.
So those are the kinds of thingsthat , people need to verify.
And with ai, , I always ask whenI'm using AI for something specific,
(06:45):
like a question that I'm asking, Ialways ask it to give me the sources.
I think that's one of the things thatI think people in law enforcement
especially that have never usedit, the question becomes, well,
how do you know that it's right?
What it's telling you is accurate?,
It the same way if the same wayyou check any kind of intelligence
you get, you verify it.
(07:05):
You
put on your
critical thinking hat and you verify it.
Yeah.
I think that also gets back on whatyou're asking it to do and 'cause I, I
see it as from my perspective when I'musing chat, GPT or another AI software
is, I am usually asking it to help mewith something maybe technical right.
(07:26):
SQL code or an Excel formulaor something to that.
Effect Right.
Obviously I am then taking whateverthey're giving me and testing it out.
And that's the verificationprocess, right?
With that.
Right.
Exactly.
Whereas
if you're getting something where it'slike, tell me what the crime stats,
(07:47):
in Santa Monica are, then you'regonna have to go through some kind of
validation process to know, get intothe source and be able to verify.
What it's reporting back asthe crime stat to Santa Monica.
Right.
The, the thing is though, theAI is a, can be a great tool.
Mm-hmm.
Especially for analysisof large amounts of data.
(08:09):
It's, it's just, you can feed it awhole bunch of information and say.
Give me a summary of this.
And or you can nowadays because,and it's only been a couple of
very short years since mm-hmm.
It became so ubiquitous.
But nowadays when you do a Googlesearch or a Bing search, you're
automatically getting AI results.
(08:31):
Mm-hmm.
Part of my, if you put this up in, inthe show notes I included a, a graphic
that I found online, and I'm sorry Ididn't put a, a source on here for this.
I think I have it somewhere.
I'll have to put it on in.
I'll send it to you in an email.
Mm-hmm.
So.
So I can do what I should havedone and include the source.
But anyway, this graphic shows theranking factors that Google and being
(08:53):
use it's right under, underneath being,it says, uses open AI technology, which
is chat GPT Technology to understandcontext and deliver on user intent.
So right there in the very first line ofthe being ranking factors for key words,
it uses open AI technology to understandthe intent of the search question.
(09:15):
Mm-hmm.
So being admits it right up front.
We use ai.
So ai, machine learning, all ofthese automation tools, they're able
to scan massive amounts of data.
They can identify connections.
For example, I, I don't really recommendit because it's not very thorough yet.
When you're doing a peoplesearch, say you're looking for
(09:37):
Jan Mondale online, you can type.
My name into chat, BT, and it givesyou a pretty good idea of who I am,
but that's me being purposely openon, on the internet because I like
to, when I do test searches, I searchagainst myself to see what I can
see, what kind of search results Iget for different kinds of queries.
Mm-hmm.
(09:57):
And so I have tested chat GPT against myown name and it gives a pretty decent.
Result.
Most of it's from my LinkedIn account.
You can tell just by looking at it.
Someone I know recently asked me for helpfinding out, verifying whether somebody
was, had a military, was a veteran,and that's what their question was.
(10:18):
And I said, oh, that's really hard.
Unless you're the person, you aren't gonnaget it from a government source unless
you are the veteran and you're trying toget information about your own record.
It's, it's pretty close.
Hold information on whether or notwhat your military records are.
I said you could try social mediasearching, but I said you are relying, you
still have to figure out a way to verifyit because you're relying on that person.
(10:40):
For example, me, if I put onmy LinkedIn account that I am
a veteran of the Air Force.
You are relying on me telling thetruth on my social media account.
Mm-hmm.
So there is that, that worrythat somebody's putting false
information up there and chat.
GPT is relying on that false informationto give you your infor your answer.
(11:00):
So you, again, you have to verify, youhave to figure out a way to verify it.
You can have a good hypothesisthat I was in the Air Force based
on my LinkedIn account, but youcan't verify that very easily.
Because unless you can figure outa way, if I were dead, maybe you
could find whether through find agrave if I'm buried in a veteran's
(11:20):
cemetery or something like that.
I mean, it's, it's just notan easy, an easy thing to do.
Especially if that person, I, I onceon social media, open source, I. Public
social media page, a person posteda picture, and I totally forgot that
that picture, just because he posted iton that day, doesn't mean that that's
(11:42):
the day that that picture was taken.
Hmm.
And so I used that as confirmation thatthat person was where he said he was.
And that was stupid.
That was very stupid.
That person did that on purpose.
Because he wasn't wherehe was supposed to be.
And so he totally fooled me, didn't fool.
Fortunately I was working with the FBIon that case, and the FBI wasn't fooled.
(12:05):
So because they had other sources,they had flight, flight data that
showed that he was on a flight whichis not open source information.
So sometimes, especially lawenforcement analysts are, are kind
of fortunate because they can relyon governmental sources that aren't.
Publicly available information toverify what might be on social media.
(12:26):
And so with ai you always have to becareful of that, but it is wonderful for
correlating and giving a comprehensive.
View, quick view ofwhat might be available.
It's also really great.
I know that we have a lot of translationcapabilities with Google search and
Google Translate and being translateopen AI can do it really fast too.
(12:50):
Just upload your text yourforeign language text.
Into an open I ai search box andask it to translate it to English.
And it's really fast.
And it'll even summarize it for you.
If you ask it, ask it to, like ifit's a foreign language journal
article or something like that.
So there are reallygood reasons to use it.
It's really, really helpfulfor threat analysis.
(13:12):
I am, I'm a cyber threat analystnow, and so it's very helpful.
I don't use it 'cause my companydoesn't allow us to use ai.
But.
If, but other companies do, thereare large companies that like that,
that people hire just to do their,to help them with their large amounts
of data, and I know that they use itsilo or authenticate, recorded future.
(13:37):
What are some of the others anyway?
Big companies that are private orcommercial intelligence companies use
open AI or, or their own, 'cause a lot ofthem have built their own large language
models and they feed the data in and they,they use that data to provide support
to the companies that, that, that arehired by them or they, hi, hire them.
(14:00):
Mm-hmm.
So some companies don't.
Allow their own analysts touse it, but they rely on other
companies that are using it.
However, those companies, I do haveto say, they're really good about
identifying what they've used AI for andwhat they've used a human analyst for.
So they're really good about that.
Yeah.
So.
So, yeah, it's definitely used.
(14:22):
It's not used by everybody.
I highly recommend if your, ifyour organization, if your law
enforcement organization allowsyou to use it, that you check it
out, see what you can find out.
It's great for multi-language stuff.
It's great for quickly finding instancesof a username across different platforms.
That's one of my, my techniquesfor finding people is a lot of
(14:44):
times people use their usernameon more than one platform.
Mm-hmm.
And so you can get pretty good results ifyou type a username into chat GPT and say,
can you find where this username shows up?
On the internet or somethinglike that, and it'll do a pretty
good job of finding all of thedifferent sites or IP addresses.
It's good for doing large scaleanalysis of tactics, techniques, and
(15:08):
procedures that cyber criminals use.
To attack organizations so you canset it up to autogenerate Alerts for
Threat Actor Activity for Spam thatmight be coming in to your organization.
Phishing detection.
Detecting questionable emailsthat are sent in order to try
(15:30):
and get you to give information.
It's, it's a very common practicefor cyber criminals to send phishing
emails to organizations claiming tobe that organization's security team.
And they want you to, they wantyou to change your password.
So, I mean, most it people are smartenough not to fall for that, but there.
(15:51):
They catch every, they catch people.
Every now and then, somebodyaccidentally goes, oh yeah, I guess
I do need to change my password.
And they put in their old passwordand their new password, and all
of a sudden a cyber criminal hasac access to their organization.
So you, you just, it's really good for.
For things like that to help youwith large amounts of information.
(16:11):
Right, and I've already pretty muchgone over the, the disadvantages.
The fact that it could be, it, the, theAI could have been fed false information.
And it's gonna give you falseinformation if it's not given good
information, and it can be biased.
Mm-hmm.
That's a, a complaint a lot of timesfrom different sides of the political
spectrum is that AI is biased by, by theimplicit bias biases of the person who's.
(16:40):
Something that annoys me as a, asa former librarian is a article I
read last week about, about Googleingesting huge amounts of copyrighted.
Books into their AI model andknowing that they will get sued for
it, but knowing that they're sucha huge company that they'll win.
Hmm.
And even if they have to pay,pay something, it doesn't matter.
(17:02):
They're such a huge companythat they're super rich and
that's what this article said.
And that, that just, you,sometimes you get bad information.
That's not necessarily wrong.
Information just shouldn't be there.
So that kind of thing.
It's, it has its disadvantages.
A lot of people worry that it's, ohno, AI is gonna take over the world.
It'll only take overthe world if we let it.
(17:24):
If, if you feed that information intoit, if you allow it to be your only
source of information, then yeah, it'sgonna probably take over your system
and, and you're not gonna get a verygood reputation for your analytical
skills if you use it to the exclusion of.
Of your own really smart, analyticalskills that no matter what,
(17:44):
you're smarter than a computer.
Because computers only areas smart as you make 'em.
One of the things that , , I uploadedwhen I sent you my show notes was
a little graphic and I just wantedto point out that if, if you do
post this in the show notes that I.
I, I fed my notes into chat GPTand asked it to make a graphic
for me, and that's what it made.
(18:05):
So you can use it forthings like that too.
Yeah.
Yeah.
And it, I've, I've used itfor that as well, and so it's
interesting how some little thingsthat'll the, it'll get wrong.
Either misspellings or the graphicwill be a little odd looking or
it's funny how it's not quite right.
Made
the, made the little computer, thelaptop computer a little too big.
(18:27):
Yeah,
yeah.
To cover covering up some of the text.
Yeah.
But anyway, that's just what, yeah.
We'll put that in.
The show notes mean outta my notes.
Yeah.
We'll put that in the show notes.
I, okay.
I, I think for, for law enforcementand for analysts, I, I think
it's really difficult to.
Take AI seriously interm in terms of osint.
(18:48):
And I think what I mean by thatis , you're only scratching
the surface when using it.
I, I would only envision using itwhere I, I'm just curious what osint,
I mean, AI on Osint on a particulartopic would kick out knowing that
I'm already probably well versed inwhat I should expect to see and Yeah.
(19:10):
But it could.
Possibly think, take you down a paththat you didn't expect, or maybe it's get
gets you a lead that you, didn't expect.
So I would think that'spretty superficial.
Like you're not going much moredown the rabbit hole than that if
you're using AI for, for osint.
Yeah, I don't think, I mean,it kind of just depends on
(19:32):
how much you wanna trust it.
I'm trying to find on my,unfortunately it's really weird.
Oh, here we go.
I did, I did a search, this was
a couple of weeks ago, not, not knowingthat this was what I was going to
do the, the podcast about, but I, I
did it.
I did a search.
And on who is Jan Mondale And accordingto Chachi pt, it, it searched two sites.
(19:57):
And one of the sites wasyour site Leap podcast.com.
Oh, the other one was, the other one
was LinkedIn, so I thoughtthat was really interesting.
Yeah.
But according to this, I am.
A cyber threat
analyst with a diverse background,including service in the military
and working in Ephesian Center.
I, I was recruited by the CIA aftermy retirement from the Air Force.
(20:19):
It doesn't say that leveragingher expertise in library science
and intelligence analysis,I mean, it, it gets a lot.
It doesn't get this very well.
Mondale, it doesn't say Jan Mondaleor, or Ms. Mondale or anything.
It just says Mondale is currentlyworking for a major retailer.
Her career is highlightedby her inquisitive nature
and extensive knowledge.
(20:39):
She even contributed to thwartinga potential terrorist attack.
I think they might have gottenthat from your show notes.
'Cause I don't think Ihave that on LinkedIn.
Additionally, she has personal interest in
motorcycles and is a memberof the Iron Bud Association.
Yeah.
That you, we talked about.
I remember that.
Yeah.
So I, I think, I think other than LinkedInthat your show notes are probably it's
(21:01):
primary, but I thought that was funnythat it only searched two sites and.
Yours was one of them.
Nice, nice.
Yeah.
Yeah.
As we wrap up , there's certainly , you'regonna have more and more talk about ai.
It's going to be a populartopic for decades to come and
certainly have gone through today.
(21:22):
Some of the.
Pitfalls and uses for, for ai In terms ofosis, just, I want just wanna give you the
last word on any advice or any, anythingelse that you want to add to the topic.
As with all intelligenceanalysis be a critical thinker.
I mean, as you can tell from whatI read about myself just now,
(21:42):
it's not necessarily always right.
Mm-hmm.
And so.
I, I just, I think you reallyhave to verify, verify, verify.
There you go.
If, if all Intel analysts would verifywhat they, what they find and not just
automatically go like I did with theguy who intentionally posted wrong
(22:03):
information on his Facebook page in orderto convince whoever might be looking that
he was where he was supposed to be whenhe wasn't where he was supposed to be.
You just always haveto think the bad guys.
Are using AI too.
Mm-hmm.
And so you've gotta be as smart as, asthey think they are, and you have to be
smarter than they are so that you aren't,you aren't falling for bad information.
(22:26):
But using it for good becausethere is really, I, I think that.
In some ways my husband's life mighthave been saved because of chat GBT last
month because I questioned everything thedoctor said and I, I really think that
that is something it can be used for.
For great.
Good.
You also have to verify.
(22:47):
All right, Jan, thank you again.
If, for the listeners, if youwant to email the show, email
us at lea podcasts@gmail.com.
You can also find us on Facebook,LinkedIn, Twitter and many other outlets.
If you have any questionsor concerns let us know.
, Until next time Jan, thank you forthe open secret and you be safe.
(23:11):
Thank you, Jason.
You too.
Talk
to you next time.
Thank
you for making it to the end of anotherepisode of Analyst Talk with Jason Elder.
You can show your support by sharingthis and other episodes found
on our website at www podcast.
Dot com.
If you have a topic you would likeus to cover or have a suggestion for
our next guest, please send us anemail at Elliot podcast@gmail.com.
(23:34):
Till next time, analysts keep talking.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
Ridiculous History
History is beautiful, brutal and, often, ridiculous. Join Ben Bowlin and Noel Brown as they dive into some of the weirdest stories from across the span of human civilization in Ridiculous History, a podcast by iHeartRadio.