May 27, 2025 • 53 mins
What does the future of Cyber Threat Intelligence look like beyond basic feeds and reports? Former CIA analyst John Doyle takes us behind the curtain of modern CTI operations, revealing how smart teams are delivering value across entire organizations.
With over 16 years tracking state-sponsored APT groups and now serving as a principal intelligence enablement consultant, Doyle explains how CTI roles are evolving to meet expanding demands. "Organizations use CTI for one of three reasons," he shares. "You've saved the company money, you're making the company money, or you're improving efficiency." This value-driven approach has transformed how CTI teams position themselves in the security ecosystem.
The conversation explores frameworks revolutionizing how teams measure their impact, including the CTI-CMM with its newly developed metrics system. Doyle also highlights unexpected partnerships forming between threat intelligence and other business units—from security awareness to HR—as threats like North Korean IT workers infiltrating legitimate companies create challenges that span traditional departmental boundaries.
For practitioners seeking growth, Doyle maps out the conference landscape from Washington DC's CyberWarCon to European events like FIRST CTI, noting that despite the industry's introverted reputation, these gatherings feature "the smartest people in the world who are super humble" and eager to share knowledge. He also details how AI is transforming intelligence workflows, enabling resource-constrained teams to operate at much higher capacity while maintaining the critical human judgment that separates great analysis from mere data processing.
Whether you're building a CTI program, looking to prove your team's value, or simply curious about how intelligence tradecraft translates from government to private sector, this conversation offers practical insights into an industry where collaboration remains the ultimate competitive advantage. As Doyle concludes, "The more opportunity we have to work with each other and grow from one another, the better off we're going to be."
Resources:
https://cti-cmm.org/
https://medium.com/@likethecoins
https://klrgrz.medium.com/
https://services.google.com/fh/files/misc/cti-analyst-core-competencies-framework-v1.pdf
Thanks for tuning in! If you found this episode valuable, don’t forget to subscribe, share, and leave a review. Got thoughts or questions? Connect with us on our LinkedIn Group: Cyber Threat Intelligence Podcast—we’d love to hear from you. If you know anyone with CTI expertise that would like to be interviewed in the show, just let us know. Until next time, stay sharp and stay secure!
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
John Doyle (00:00):
Everybody there is just the smartest person in the
world and super humble.
Rachael Tyrell (00:04):
Hello and welcome to Episode 7, Season 1
of your Cyber ThreatIntelligence Podcast.
Whether you're a seasoned CTIexpert, a cybersecurity
professional, or simply curiousabout the digital battlefield,
our expert guests and hosts willbreak down complex topics into
actionable insights.
On this episode of Season 1,our host Pedro Kurtzman will
chat with John Doyle, who hasover 16 years of experience
(00:26):
working in CTI, digitalforensics, cyber policy, and
security.
Pedro Kertzman (00:57):
John, Thanks so much for joining the show.
It's great to have you here.
John Doyle (01:00):
Hey, Pedro.
Thanks for having me.
I'm really excited about this.
Pedro Kertzman (01:02):
Awesome.
Usually, I start asking theguests about their journey into
CTI.
Would you mind walking usthrough that,
John Doyle (01:11):
Yeah, for sure.
My story, I feel like, iscomical, much like a lot of my
colleagues in this field, how wealmost got here by accident.
My journey kind of starts atuniversity.
I was working...
2008 timeframe for our GeekSquad, or the equivalent of our
Geek Squad on campus.
(01:31):
And at that point, I had spenta few years, and that really
kind of got me into computers alittle more than I had
previously.
So there was that dynamic.
I ultimately went to gradschool for information security
and also picked up this degreein national security studies,
not realizing that kind of wouldposition me well for
(01:52):
understanding the geopoliticaloverlay of events that happen
and how there's cyber actionsthat are interrelated.
Spent a decade thereafterworking for the CIA as a threat
intelligence analyst focusedspecifically on cyber threats.
And then I eventually made myway to the private sector.
(02:14):
And in the private sector, Iwas working and working for
Mandiant and I also moonlight asa SANS instructor for the
threat intelligence course.
So I've kind of I've kind ofgone an interesting route, one
that I feel would resonate a lotwith people who used to work
for government, who kind of cameout and transitioned into this
(02:35):
field, and one that I think isbecoming a little more
non-traditional in the pastdecade or so, as the really
juicy data for threatintelligence analysts no longer
lives in government spaces andclassified sources, but actually
lives in victim environmentsthat incident response firms and
(02:56):
managed defense providers orMSPs actually have access to.
Pedro Kertzman (03:00):
That's interesting.
I always thought that stillNowadays, military intel
experience would be superdesirable, for example, on the
CTI field.
But it's interesting to knowthis shift.
I never thought about that, butit makes sense.
John Doyle (03:19):
Well, there's two parts to it.
I think you hit on aninteresting first bit there when
you talked about the skills.
So one of the things you get inmilitary intelligence or even
civilian and law enforcementintelligence is kind of grounded
tradecraft.
So you've got to be able tocommunicate.
communicate well, you got to beable to do research well that
can be thoroughly backed upbased on this or that data
(03:40):
source.
So you learn kind of the basicsof research and it gets
reinforced and then you learnhow to communicate it over time
to stakeholders.
So you're learning analyticalprocesses, which is incredibly
helpful irrespective of what jobyou have.
But the cyber element is what'skind of or at least the sources
(04:01):
of information is where we'veseen kind of a change happen in
the last decade.
Pedro Kertzman (04:06):
there you go yeah the other day i was talking
to somebody that they mentioneduh something like that uh the
intelligence knowledge shouldcome first than the cyber
knowledge you can combine thecyber into intelligence but if
you don't know how to dointelligence and you just have
the cyber knowledge then youkind of need to learn from the
get go a new trade craft becauseintelligence again even though
(04:29):
we leverage cyber security uh ifyou just do for example,
malware analysis, but then youneed to learn how to connect the
dots into a moreintelligence-driven type of
approach.
Does that make sense?
John Doyle (04:43):
Yeah, I can see that.
I can most certainly see that.
I think, I mean, if we back upa little bit and look at almost
the start of CTI, you had twodifferent audience types and two
different skill sets that wewere trying to speak to.
So you had the incidentresponders and the SOC analysts
(05:04):
doing kind of threatintelligence organically to make
a determination as to like, isthis bad?
Should I take an action on it?
Does it require immediateattention?
Can I build out a playbook?
But that's different thanspeaking to like an executive
level audience where they'relooking at a more strategic
overlay.
They're trying to identifycomplexities and trends in cyber
(05:28):
threats and how this relates totheir organization's risk
posture.
And we're actually seeing,definitely seeing kind of more
alignment happen and moreconvergence happen over the last
decade.
But it's just two differentskill sets.
I mean, on the former, you'vegot kind of a digital forensics
(05:48):
incident response background,cybersecurity background, moving
into a more higher levelservice support construct.
And from the other side, youhave traditional intelligence
analysts who are used toproviding insights in a way that
resonates with differentaudiences at a high level so
they can make informed decisionswho are then having to learn
(06:11):
the more technical and the morenuanced aspects of this field.
Pedro Kertzman (06:15):
Got it.
John, awesome.
And so you mentioned a fewaspects, important, very
important functions within a CTIteam as a whole.
Any other changes that you'veseen in the past possibly few
years within how those roles areevolving or changing based on
(06:39):
either the maturity of theindustry itself or from external
factors, for example?
John Doyle (06:47):
Man, you asked a loaded question there.
Yeah.
Let me think about this.
Yep.
We've seen...
a few different evolutions.
So there are more companiesthat have access to firsthand
information and also secondhandinformation that is really
(07:08):
important for building out anunderstanding of adversaries or
threat activity groups.
So on the vendor side, there'smore vendors with useful data
that exists that is happeningconcurrent to a high level
demand for threat intelligenceto help inform decisions whether
(07:31):
it's at the board level theexecutive level the cyber
security program manager levelor even the threat intel manager
level then there is the skillsso the demand from other
internal stakeholders is thenpredicating the need for
individuals on the teams to bebroader and deeper in their
(07:58):
specific craft.
And that's causing, dependingon the organization, the
resource, the budget, all ofthose kind of administrative
constraints and kind of growthprojections, causing individuals
to have some level ofspecialization, but also having
the need to have a baselineskillset or at least a
(08:19):
functional understanding of howwhat i do compares to what you
do and how it's complementaryand how we can then use that
together to work effectively todrive better outcomes
Pedro Kertzman (08:32):
got it okay that's uh that is interesting
you mentioned like part of theEvolution now, the way I
interpret it, it's bringing morevalue to all levels of the
organization and not anymorejust the SOC or just that
particular department.
(08:53):
Actually, I was provokingpeople the other day that threat
intelligence is not just threatfeeds and reports anymore.
You can have way more emissiondecision making.
You can use that CTIinformation to do way more than
(09:15):
just playbooks with feeds andunderstanding what's going on
with the reports and things likethat.
So it makes sense.
It's interesting and it's goodto see the industry overall
shifting in that direction.
One of the struggles I stillhear a lot from CTI folks is
(09:36):
being able to prove the value oftheir department into the upper
management levels.
So they don't get as impactedor sometimes with layoffs and
stuff like that, but even, hey,I can help you with this.
If you have like a decision todo, I can bring Intel and maybe
(09:56):
we'll quote unquote, enrich thedata that you have to take that
decision.
Sorry, go ahead.
John Doyle (10:09):
No, no, no, no.
I mean, you're hitting on twothings I think a lot about.
One, what's the best valueproposition for an internal CTI
team?
And then what is the valueproposition for intelligence
vendors who may service internalCTI teams?
And this is something that wheni'm working with clients who
(10:33):
are building their cti programsor trying to mature it these are
all decisions that they'retrying to make these are all
complex situations they'retrying to navigate is how do i
convince the business that wehave the ability to provide
proper level support to servicethese needs that are measurable
(10:54):
that actually show impact bothat the bottom line of the
company, but also showsefficiency gains.
So it's...
My boss at one point in timesaid something to me, and I
don't think that he meant it aslike this wise sage advice that
was super impactful, but itactually was.
He's like, look, look, John,like organizations use CTI for
(11:17):
like one of three reasons.
One is you've saved the companymoney and money could translate
to like brand reputation lossor a whole host of other kind of
things that are associated withrisk.
It's making the company moneyor you're improving efficiency
in some way, shape or form.
I was like, damn, that wasgood.
I need to, I need to take morementorship from you.
Pedro Kertzman (11:42):
Yeah.
Mentors.
Right.
So, so valuable.
I feel you.
Yeah.
Some quotes just have thatpower to stuck on our heads and
we will keep thinking about themand using as a guideline to do,
you name extra work research,something like that.
John Doyle (12:00):
There is like one more thing here.
Have you heard of the CTI-CMMbefore, the CTI Capability
Maturity Model framework thatexists?
Pedro Kertzman (12:11):
I heard of it, haven't had the chance to deep
dive into it yet.
John Doyle (12:18):
Okay, I'm going to give it like a 30 second plug
here.
So it's a model that's designedto effectively demonstrate that
value prop across differentstakeholders.
So here is incident response,here's SOC, here's hunt, here's
red team, here's third partymanagement, et cetera.
What is there?
organizational function do andhow can CTI support it?
(12:43):
And then it breaks it down overa spectrum of maturity levels.
What they just done in the pastmonth is actually created
metrics.
So I'm an internal CTI team.
What are good metrics for me tomeasure against if I am
supporting incident response, ifI am supporting purple team, if
I am supporting red teamexclusively to help with,
(13:06):
because it's hard, right?
Metrics, creation, is notsomething that like me as a
deeper practitioner have beentrained to do, or even like
somebody coming into amanagement role.
They've not gone to, you know,university or taken professional
training on like, how do Icreate effective metrics?
But my boss is asking me tocreate something, so I have to
create something.
(13:26):
so this framework is actuallydesigned to bridge a lot of
these gaps to help with thatvalue prop for intelligence
teams as a whole so like if ididn't have that organically i
don't have to create it fromscratch it now exists it's
something that i can take andpull from
Pedro Kertzman (13:41):
that's amazing that's that's super important to
how to to sell the value ofyour of your cti team uh and
maybe um i don't knowbrainstorming moment um do you
think cti could go uh and offeras well value to um some
(14:03):
companies used hr to manage uhcyber security awareness program
or you know work handy handwith it for that but do you
think for this type of programcti could also offer some value
John Doyle (14:22):
absolutely I think the closest partner that a CTI
team could have besides incidentresponse is security awareness.
And that's for a host ofdifferent reasons.
So one of the countries I usedto track a lot was North Korean
cyber threats.
So all of the groups associatedwith that.
(14:43):
And in the last year, One ofthe trends that we started to
see emerge, and it's not netnew, but it's kind of net new in
the public's eye, is thisnotion of North Korean IT
workers applying for roles atbespoke companies or even big
name companies, which hasfundamentally done something
(15:06):
quite unique, I think, in thisfield.
It's allowed us to not justlook at threat activity.
In part, some of these actorsare using their access for
threat activity.
or providing the accessremotely to cyber operators.
But it now lets us actuallybridge the gap because this is
something that HR needs to knowabout to be able to screen out
(15:28):
North Korean IT workers that aretrying to work for their
company.
It brings in the notion ofinsider threat and it really
kind of expands the aperture toshow what that value prop could
be for threat intelligence tosecurity awareness training, HR.
identity and access managementand like a few others and it's
that's not the only case toowe've been seeing but we've also
(15:50):
seen like i'll admit i was abit of a naysayer when it came
to you know using the dark webto surface things that are
useful for cyber threat researchBut like the notion of identity
intelligence is yet anotherarea that's starting to prop up.
So leaked creds, customer PII,internal PII that's leaked, like
(16:11):
all of that's really important.
And I think this just speaks tothe evolution of the industry
in the evolution and almostremit in things that are
expected of a CTI team to beable to cover, let alone
knowledge of like exploitationof edge devices and some of
these other things that weremore like, whoa, that's very
(16:33):
unique, specialized knowledge,but now it's almost expected.
Pedro Kertzman (16:37):
Yeah, no, that's great to, let's say, brainstorm
that with you.
I'm going around some localconferences here to try to give
real use cases andthought-provoking things to
people that CTI should go beyondjust threat feeds and reports.
(16:58):
you're basically short sellingyour value if you're just
focusing on that, right?
So it's great to hear that I'mnot alone.
Yeah, no doubt about that.
That's awesome.
Man, and you mentioned aboutthe CTI maturity model as well.
(17:19):
Any other interestingframeworks that we've seen
around that perhaps noteverybody is using and they
should use more or explore more?
Anything around that?
N
John Doyle (17:32):
I'm a little biased for what's going to come out of
my mouth as the author of it,but I'm going to do my best to
give it as an objective of asell as possible.
Pedro Kertzman (17:43):
Fair
John Doyle (17:43):
enough.
So a few years ago, we had beengetting asked a lot of
questions on the Intelconsulting side about, well,
what's the right composition fora CTI team?
What are the right skills?
What are the right backgrounds?
Who should I hire?
How should I hire?
It's all sounding veryfamiliar, isn't it?
So I kind of went out there andI'm like, well, let's see what
NIST has to offer.
NIST is a standards body.
(18:04):
They do a pretty good jobacross the board at putting
useful information out there.
Let's see what they have.
And I came across NIST SP800-181, which is the NICE
framework, the NationalInstitute on Cybersecurity
Education.
Oh, cool.
Fantastic.
Let's drill down into this.
(18:24):
And as you start to see all ofthe different role profiles that
would be part of acybersecurity program, CTI gets
parsed out almost into three orfour different categories.
You've got a collectionsmanager, you've got a threat
warning analyst, you've got soand so forth.
And as I looked at this, I waslike, I'm just curious how much
(18:49):
government influence there washere.
And whether there was anybodyfrom private sector who actually
weighed in on this.
So, you know, I ended upconnecting with two individuals
who were leading the NIST NICEproject, and we had a very
cordial conversation.
It was a very goodconversation.
And they're like, yeah, youknow, you're right.
(19:12):
A lot of this wasgovernment-backed.
Like, we kind of leveraged themfor, like, what does right look
like?
I was like, cool.
I work with threat intelligenceteams that are, like, six
people max you mean to tell methat one of their six people is
just managing collections andlike that's all they're doing
now I get it if you're workingin like a several hundred person
(19:35):
intelligence community thatthese are distinct roles but you
mean to tell me that there's anexpectation being made here
that like one person is doingjust this job exclusively like
yeah okay to each their own umSo, so really kind of prompted
me to come back to the table andI asked her, I was like, well,
(19:56):
I've got a lot of ideas herefrom what I'm seeing.
Is there a way we might be ableto like merge this from like
your knowledge and skills andabilities you've broken out or
your KSAs you've broken outacross the different, um, the
different role profiles.
And they're like, yeah, butwe're actually thinking we might
switch the model up a littlebit and it's going to actually
potentially break things.
(20:16):
So like, if you want to go outand do something on your own and
create your own framework,that'd be, that'd be cool.
Like we'd love to collaborateor at least have insights.
I was like, yeah, I could dothat.
So this led to the creation ofthe Mandiant CTI Analyst Core
Competencies Framework, which isbroken out into four different
pillars.
(20:36):
Those pillars functionally canbe broken into two categories.
The first category is softskills and professional
effectiveness.
And the second category istechnical acumen and threat
knowledge.
So we enumerate 182 differentknowledge and skills and
abilities inside of thisframework itself, where if
(20:58):
you're interested in justjoining the field or like you've
been in it for a while andyou're like, all right, what's
next?
You could actually rateyourself.
So there was kind of threedifferent cruxes for why we
created it.
The first was self-inventoryevaluation for personal
development and professionaldevelopment.
Second was to helporganizations with hiring
decisions because it's like,man, what do they need to know?
(21:20):
Like, I feel like this guy justneeds to know more than the
MITRE ATT&CK framework and whatit's used for.
Like what are those otherthings?
So it was designed to alloworganizations to almost lift the
respective KSAs to include intheir job requirements.
And then we had a third onetoo, which I kind of bled into a
little bit already with thedescription of the other two,
(21:41):
but it's for that evaluation.
How do I evaluate where mystaff are and how they can grow?
So creating almost like a teamreport card across the different
areas based on the roleprofiles.
So that's one that I think Ithink it works quite well.
It carries the Mandiant namebecause I was being paid while I
(22:01):
developed it, but it was donein coordination with the private
sector.
A bunch of my peers who are inthe industry here, either at
vendors or working simply inprivate sector and also in
public sector too, they wereable to weigh in on it.
So that's, I think, one of theframeworks that maybe got me
invited to work on this cti cmmproject okay um so it's it's
(22:28):
been good i mean the the thegenesis of a lot of these
projects is to help fill gaps inindustry whether it's showing
up knowledge or helping programsbuild to better because we're
all in this together right it'sa small industry
Pedro Kertzman (22:42):
man well yeah that's great let me uh digest a
little bit here and pause for asecond.
I like it.
And by the way, just mentioningabout the Sense CTI conference,
it reminded me, any other goodCTI-related conferences you had
(23:05):
the chance to attend?
Any values you saw on this orthat other one?
Anything to comment on that?
John Doyle (23:15):
I love this.
I've been thinking a lot aboutthis too.
Yeah, there's a whole handfulof them and I think it speaks to
kind of both the specializationand the growth of the industry
in different geographies.
Some conferences are bettersuited for generalist audiences
(23:36):
or those who are kind of entrantinto the field, maybe upwards
of three, four years.
Some are better for researcherswho have been around for a
little while longer looking toconnect.
And conferences usually takeone of two forms.
One, it's open and you pay toget in, or sometimes it's free
for live streaming.
The other is kind of like thisclosed trust network where you
(23:57):
need somebody to vet and verifywho you are and that you're
trustworthy because informationbeing shared might be shared at
a certain sensitivity level likeTLP Red.
Only the people there can talkabout it.
Maybe we start with the SANSCTI conference since you already
mentioned it.
That one happens in January.
It's in DC.
(24:18):
DC, Alexandria, Arlington area,usually.
It's been a good one.
It's been one of the longerstanding ones that exists.
Free to stream, nominal fee toshow up there in person, usually
maybe about 250, 300 show up inperson.
It's a really good way tonetwork.
And maybe I back up for asecond.
(24:39):
There's different valuepropositions for people who go
to conferences at differentlevels.
And it's a little weird too,because I feel like a lot of us
on this field, we tend to beintroverts.
So like our social battery isdrained and like going up to
somebody like thought of me andlike, hi, I'm John, who are you?
And what do you do?
And who do you work for?
And like the 20 questions, it'sjust intimidating.
Pedro Kertzman (25:03):
I'm happy to know it's not me.
It's only not me.
Oh man.
I go back to the hotel.
I'm like crashing, crashing.
John Doyle (25:10):
Yes.
Yeah.
No, but like, that's, that'snormal.
And like, that's, it's funny.
Cause my junior analyst, Italked to a lot and I'm like,
you just gotta put yourself outthere.
Like at some point, everybodylike we have this shared frame
of reference where it's likeshared experiences so like if
you're being a little sociallyawkward guess what a lot of us
are socially awkward so it'sfine but we're in our own heads
(25:31):
most of the time where it's likei don't know but what if i said
what if i sound silly and i saysomething that i shouldn't like
whatever
Pedro Kertzman (25:37):
everybody
John Doyle (25:38):
like we're our own worst critics so it's fine yeah
So the SAN CTI conference isreally good for actually
learning.
They actually did a split tracklast year and the year before
where they had kind of like newto cyber.
So people who are kind ofgenerally in that first three,
four year bucket.
And then they had like a notlike they didn't call it
(26:00):
anything special, just like thisis like the other track that we
have.
It's kind of cool because I sawa lot of individuals who were
in person who had been seasonedpractitioners for like a decade
plus stop in for some of the newto cyber talks and they have a
whole Slack channel where peoplecan ask questions and bounce
(26:21):
ideas off of each other.
Like I saw some of the moreseasoned analysts and
researchers actually answeringquestions that were being asked
during the presentation in theSlack channel.
So it's like a way to kind ofhelp mentor and build people up.
So I'm actually a big fan ofany conference that has a
Discord server or a Slackchannel that's already kind of
(26:41):
pre-established to allow forthat kind of growth amongst
peers.
So that's, that's San CTI.
There's like a few that arecoming up soon.
There's a cyber crime onecalled sleuth con that's coming
up in June.
It's not the only cyber crimeor underground type economy one.
So like team, team comry rise,it's in three different
(27:03):
locations to cater to different,uh, analysts in different
geographies.
One is in, um, Singapore forthe APGA region.
One is in Europe and one is in,I believe California.
And they happen, um, you know,at different, times throughout
the year.
Those are good ones.
You've got SleuthCon, which iscyber crime focused.
I talked about that.
(27:24):
The EPT or nation state versionof that is CyberWarCon, which
takes place in Washington, D.C.
in the fall.
But there's more.
There's a whole host of otherones.
So for the EU segment or eventhe UK segment, I'll kind of
include them as adjacent andcombined for this.
You've got cyber threats, whichthe UK NCSC puts on.
(27:46):
You've got Virus Bolton, whichis actually taking place in just
a few weeks.
You've got First CTI, which isusually hosted in either Berlin
or Munich.
Then you've got TIX, which isout of the Netherlands, the
Threat Intelligence Exchange.
And I feel like I'm missing oneor two there, but I feel like I
(28:06):
hit on most of the high-levelones.
Oh, and then there's some ofthe private closed ones, which
include like LabsCon andPivotCon.
And Woo, and maybe a fewothers.
Pedro Kertzman (28:20):
Cool.
And any of those conferences...
Would you say it's like amust-go for seasoned CTI guys?
Like a must-go if they don'tknow yet?
John Doyle (28:31):
Yeah.
Cyberwarcon and Sleuthcon...
Sleuthcon's kind of new.
It's only been around for a fewyears.
It used to be Brunchcon,because it was the day after
Sleuthcon, but then it got kindof parsed out as its own thing.
You end up meeting...
a lot of really interestingpeople at both of them who have
(28:52):
been practitioners in the fieldfor a very long time.
And I mean, one of the thingsthat resonated with me at both
of those conferences, and maybethis is more representative
reflection on the field, is likeeverybody there is just like
the smartest person in the worldand super humble.
Everyone's willing to talk toyou about anything and help kind
(29:14):
of mentor and grow.
So the networking dynamic youget from either of those
conferences is wild.
Pedro Kertzman (29:21):
That's so nice.
John Doyle (29:21):
But of course, the drawback is you're in the U.S.
around Washington, D.C.
So for some people, that's costprohibitive if they're trying
to travel internationally, whichis why I really like the
evolution and advancement ofthese kind of EU-type equivalent
conferences to make it moreaccessible, this field to be
more accessible.
(29:42):
Because you still get someresearchers who speak at the
conferences who come from theU.S.
or will come from EU or willfly in.
So it's not like there'sdiminished value in any of them.
It's just they're kind ofsometimes different flavors of
the quality you get.
But the people who go, topnotch across the board.
Pedro Kertzman (30:06):
Yeah, that's a great point.
I think the audience willchange, but the quality of the
speakers might be similar,especially for the conferences
paying for flights and stays andall that.
So, you know, if people want togo there and have their session
or keynotes, you know, it'seasier for them.
But it's not as, like youmentioned, prohibitive for
(30:27):
people.
thousands or hundreds of peopleflying from one continent to to
the other yeah i think blackeven black hat now has like a
apj uh europe and and us ofcourse
John Doyle (30:41):
yeah and if you're looking for kind of more local
cyber security conferences so ifwe extract a little bit outside
of cti the b-sides conferencesare great to go to
Pedro Kertzman (30:49):
oh i love
John Doyle (30:50):
them there's local chapters of these sites just I
mean, throw a rock in any whichdirection, you're going to find
a local chapter.
Pedro Kertzman (30:58):
Oh, yeah.
Honestly, I think any mid-sizedcity in North America, I would
say, probably has a B-Sides atthis point.
Or a metropolitan area.
Yeah, yeah.
Awesome.
Yeah, so changing gears a bit,you mentioned you used to work
for the CIA.
(31:18):
Of course...
you know, details arecompletely confidential, but
generically speaking, anyspecific part of the job, how
used to be like any part of theroutine you could mention and,
uh, any insights from that, um,and, and maybe from a broad
audience perspective, thingsthat could, uh, people could
(31:42):
leverage from the overall, uh,industry.
Unknown (31:46):
Yeah.
John Doyle (31:46):
Yeah, no, that's fair.
I'm not sure that I have anythat I can publicly share, but
there's a lot of things thathappen behind the scenes that do
make their way into the news.
And likewise, the news cyclehelps prompt some investigations
into different actor activitiesfor analysts or analysts.
(32:07):
So we're always looking forthings that are related to
whatever our focus is.
So open source being one ofthem was absolutely something
that I was looking at.
And I was like, oh, this groupis saying this.
And then for us, maybe therewould be a policymaker in
Washington, D.C.
who would read this particularnews thing and ask a question
(32:28):
like, hey, what does this mean?
Should we care about it?
And then the question wouldfilter its way down to the
respective organizations andwe'd answer it.
We'd have the opportunity tokind of weigh in, add some
ancillary information about it,maybe the actors, maybe the
groups, maybe the types ofactivity, and not just help them
(32:50):
with the immediate ask, butalso anticipate what other
information they might need.
And this relates to this trendthat we're seeing, which then
allows for some opportunityanalysis on top of that to help
make life harder on threatactors, or clamp down on threat
activities, or help guide youknow, The funding streams of
(33:16):
different initiatives or justotherwise kind of highlight gaps
and areas where the policycommunity could plug in to
really help fill them.
So in a lot of ways, it waskind of a really cool
environment that I found myselfin that I didn't think a defer
practitioner had that type of aflair or could have that type of
(33:38):
a national level impact.
But yeah, kind of here we are.
It is unfortunate that theagency is very close-held about
some of their successes thatcome out because I think it'd be
really cool for some of thestories if they did make their
way public.
Pedro Kertzman (33:54):
Yeah, share best practices.
I would imagine things of thatnature, right?
Yeah, it's a complicatedtrade-off, right?
Because you don't want toexpose those best practices
because people can, on the otherhand, could leverage that to
maybe bypass them.
But it's interesting to thinkthe line, right?
(34:21):
Where's the risk of sharingthat, but also the risk of not
sharing that with a broaderaudience that could, I don't
know, become targets, forexample.
So it's a tough decision tomake, I would imagine.
John Doyle (34:37):
Yeah, so on the one hand, the stories about what was
worked and what the results forthe impact might not come out.
But the tradecraft, I think, iscoming out for threat research
because we're seeing, and wehave seen for the last decade
(34:59):
plus, a lot of really talentedindividuals leave government to
go work for private sectorvendors or individual companies.
Then you've got the ISACs orother trusted communities or
these conferences where it'sopportunities to share best
practices in how we go aboutdoing tracking, how we do
(35:22):
alignment to stakeholders, howwe find signals in the noise
that are interesting and buildthat intuition.
There's I feel like a lot ofthat is out there.
A lot of it's becoming morepublic knowledge, certainly a
lot more than like a decade agoor more, because of the high
(35:45):
prevalence of people who haverotated out from government
space, military intelligence,civilian intelligence, law
enforcement.
And honestly, I think that'sprobably done in insurmountable
amount of good for buildingresilience for building cyber
(36:05):
security for a bunch oforganizations that would
otherwise be victimized um thatat the time was probably
scrutinized like you're leavingus to not work on the mission
you've lost like you you've lostyour your vision you've lost
your perspective it's like nahnot really i'm still getting
mission impact so Overall, Ithink what was a bad news story
(36:29):
for governments with theattrition turnover actually
turned out to be a really goodthing for bolstering this
industry and bolsteringcybersecurity resilience for
organizations to help thembounce back and detect
ransomware to help.
to help foster public-privatesharing, to help grow the
(36:52):
overall security posture at thenational and at the economic
level, right?
So, I don't know.
You got me on a rant there.
Pedro Kertzman (36:59):
No, this is actually an excellent
perspective.
Thanks, John.
I appreciate it.
It makes total sense.
I think at the end of the day,everybody is benefiting from
that perspective.
pool of amazing resources thatthe government in some shape or
form was able to provide andtrain.
(37:20):
You know, like you mentionedbefore, right?
This is a team sport.
We can only win this together.
If you have like a superknowledgeable, insane expertise,
part of the equation let's saythe government from municipal
(37:44):
states uh federal level but thenthe companies you know are
falling apart from a cybersecurity standpoint then you
start getting problems with theinfrastructure manufacturing and
then as a whole it's just notgoing to be positive to to
anybody right anyhow um Youmentioned Isaac, and maybe
(38:12):
piggybacking a little bit on oneof our previous topics, do you
see the Isaac role,quote-unquote role, is also
evolving?
Some of them, or are they...
primarily focused on sharingthe more traditional forms of
(38:34):
CTR, like feeds, reports, or yousee the role of the ISACs also
changing over time?
John Doyle (38:45):
That's a tough one.
So when we say ISACs, what wemean is information sharing and
analysis centers, ISAC being theacronym for it.
Every industry kind of hastheir own ISAC or ISO.
For our purposes, let's justuse the word ISAC to avoid
technical distinctions.
Every industry owns andoperates the ISAC model just a
(39:08):
little bit differently.
So the standards and practicesin play The membership terms and
service for the healthcare ISACversus the retail ISAC versus
the financial ISAC versus the ITISAC, you know, so on and so
forth, are all going to vary.
So you'll have differentstandards levels for each.
(39:29):
I'm seeing, so I work a lotwith the Healthcare ISAC and a
few others that I won't name, toreally kind of help bolster
their capabilities.
And you're right, historicallyit's been, well, maybe we just
share out a bunch of IOCs.
Some of these ISACs areactually looking as part of
(39:52):
their membership criteria forthe participants, the industry
partners that are operating inthat sector to share with them
threat insights to then shareout at a sensitivity level.
So whether that's TLP Amber orTLP Amber Strict that says us
members here, acrossorganizations, we can use it for
(40:16):
the intent purpose of huntingthis type of adversary activity,
or because we're trying todetermine whether or not this is
a campaign.
So something has come in, atipper has come in from an
industry partner, and they say,hey, we're seeing this
proactively.
Is anybody else also seeingthis type of activity?
So in a lot of ways, the ISACsend up being a nice early
(40:38):
warning system to determinewhether there's a campaign of
significance, and chances arethat that information is
probably also being shared inother closed channels too.
But there's this push and pullmodel.
This push and pull model, andI'd actually go out on a limb
and say that we are seeing anevolution over time for getting
(41:01):
vendors and other researchers inthere to help brief the ISACs
at like their annual orsemi-annual conferences up on
tradecraft.
Or what does best practiceslook like?
But that's going to be...
ISAC to ISAC, so I can't make ageneral statement there because
of the way they're owned andrun.
(41:23):
But I have been pleasantlysurprised in the last few years
with the ISACs that I've beenworking with to see them grow,
to see them take on board andtry new things or have a
centralized threat intelligenceplatform that can act as a push
and pull model that has reportsthat are provided with those
different classificationtaglines to allow them to be
(41:46):
able to share things back andforth
Pedro Kertzman (41:48):
awesome and and john one of the things we hear
probably every single day nowmany times it's uh ai uh you see
any um kind of good usage orshifting uh across our industry
if you will uh when it comes tothe usage or where ai could help
(42:12):
us uh um on any directions youcan think, from a detection
standpoint, LLMs, you name it,any sort of AI over there
helping the CTI industry?
John Doyle (42:28):
Yeah, maybe I'll start with a quick story.
So I was asked this about ayear and a half ago by some
colleagues.
We were brainstorming, and thenagain by a client.
And then at a workshop, I kindof put on my slide that says,
AI.
What is it good for?
And then I just put up a bunchof different high resolution
(42:51):
fantasy images.
And I go, it's great forcreating characters for your
Dungeons and Dragons campaigns.
And it got a good laugh.
But it's true.
There's a lot of like-minded orlike culture type of like nerd
culture here where we've all gotour own kind of hobbies and
things.
And image generation isabsolutely one of those.
Go to find out with things likeMid Journey and Claude and
(43:14):
other kind of models there.
But as far as CTI specificapplications, applications.
If you look at the intelligencelifecycle, all five phases of
it, there is application for AIin all of them.
And when I say AI, I'mspecifically honing in on AI is
a tool that's used by humans.
(43:34):
AI is designed to augment ourcapacity and our capabilities.
So it is something that's usednot by itself necessarily, but
to help us with things like datatriage, like looking at large
scale data sets or leaks of dataand being able to surface
insights pretty quickly, a lotquicker than the human would be
(43:57):
able to.
It helps us with standardizingreporting.
I use AI almost every day forwriting things like, actually, I
just used it yesterday.
I said, hey, I need to write apair bonus for this guy.
He did the thing.
Please include this, this, andthat, and put it in a formal
tone that highlights thesedifferent attributes that I know
(44:19):
are going to be useful forhelping maybe get him promoted.
It spits something out for mein about 30 seconds that
otherwise would have taken me 15minutes to write.
And even if it gives you the80%, it's a really good starting
point.
When you start layering on topof that kind of rag, the
(44:40):
augmentation for it to gooutside of its closed data
sources and go doing deepresearch, it really kind of
shows a value add prop.
Now, kind of the drawback withthat is taking anything that the
AI gives you as truth withoutdoing vetting and validation.
So for us as analysts, I feellike a A lot of more senior
(45:02):
analysts are using this almostlike a search engine, like on
steroids.
And a lot of the more junioranalysts are just taking it and
saying, all right, this isgreat, we're done.
So like that critical thinkingand that trust but verify
mentality is quite important.
But as far as producinginformation in a quick way,
doing triage, outputtinginformation in a standard
(45:22):
structure, like please extractall of the IOCs from this
particular report and map themto MITRE ATT&CK.
it does that pretty quickly.
Oh, and please put that in atabular format so that I can
take this table and copy andpaste it into like a spreadsheet
or give me this in a CSV file.
It's really good at doing taskslike that.
(45:44):
Things that otherwise wouldhave taken us a lot of time to
do the manual curation of andtransformation.
It can do that pretty quickly.
Speaker 01 (45:51):
I've
John Doyle (45:52):
used it a little bit for the vibes coding scripting.
It works fine enough.
I'm just right.
I'm not a developer.
I can write a few scripts.
It can write the scripts inPython a lot better than I can
though.
Anytime I need to do a parsingexercise or I need to link
datasets together, I always justtake it at face value and
evaluate like, is there anythinghere that I need to change?
(46:16):
I will review the code base,but ultimately it's still the
human in the loop.
I don't know that we're gonnalose the human in the loop, but
boy, are we certainly becoming alot more productive as a result
of it.
So it's cool because in a lotof ways, AI is letting those
resource constraint CTI teams oflike two or three people really
operate at the level of like afive or six person team.
(46:37):
If it's used right, if it'sallowed in the environment based
on the risks that have to beaccepted for it, then there's a
whole host of otherconsiderations too.
But at the same point, it'shelping kind of bridge the gap
and allowing us to support thesedifferent stakeholders in a way
that historically we wereresource constrained and
couldn't.
Whereas today, it is reallykind of transforming the way we
(47:04):
do intelligence.
as a technology enabler but notas a replacement if that makes
sense
Pedro Kertzman (47:14):
100% I agree with that it brings me to a
point whenever we're trying tolet's say learn more about CTI
and I'm not necessarily talkingabout the feeds the latest
threats IOCs and reports and soon You have any go-to source to
(47:40):
learn more about the industry ingeneral outside of conferences
or new frameworks out there,things related more on the CTI
holistic approach, any go-tosources for it?
John Doyle (47:59):
Katie Nichols and Andy Piazza, both independent of
one another, pulled together toblog posts on their medium
sites.
On their medium sites, it'ssomething like the Newcomer's
Guide to CTI, really designed toprovide a lot of resources on,
well, what does seminal thingslook like?
(48:20):
What are some of these keyresources to understand and
really get into this field?
Less so on what CTI does, butthat's implicit and kind of
covered indirectly as part ofthat.
There's also...
the SAN CTI Annual Survey.
Are you familiar with that one?
No, I don't think so.
So it comes out every year.
(48:42):
They've been doing it now for ahandful of years.
It really gives goodperspective not on the vendors,
but on where industry trends arecoming and going.
So it is volunteer by nature.
They usually put like a two,two and a half month call out to
get data for people to take thesurvey to fill in, you know,
(49:04):
exact thing you would expectfrom it.
Like what industry are you in?
Like what's your, you know,average years in or whatever.
And like, capturing some kindof meta statistics that could be
used for vetting.
This year's one, I think, isdue to come out in about a
month.
I would be surprised if wedidn't see the inclusion of AI
in it, if we didn't see theinclusion of these different
(49:27):
like intelligence sources andvendors coming out.
So like the identityintelligence, for instance, like
that sounds like something thatI've been seeing kind of the,
you know, Intel 471s, theflashpoints and others really
kind of digging into dark webmore and having like the
recorded futures, like add adark web module or Palo Alto's
(49:47):
like add that as part of theirfeeds.
So like, I would be verysurprised if that, know wasn't a
trend that they tried tocapture in it too so like what
are those data sources beingused by internal cti teams
beyond like internal internaltelemetry um so it's it's really
good for kind of providing alay of the land and i think
(50:08):
that's what um that's where iwould go if i'm trying to get
kind of holistic capture atleast a quick snapshot of what
the industry looks like
Pedro Kertzman (50:16):
Oh, awesome.
Great information.
Thank you.
Make sure people listening thatyou check those medium sites
for more resources.
Excellent.
John, any final thoughts?
John Doyle (50:28):
Maybe we end on an uplifting note.
We're better together.
So this whole field, I feellike, is predicated on a bunch
of smart individuals who want todo the right thing, who want to
impose costs on theadversaries.
So the more opportunity we haveto work with each other and
grow from one another, thebetter off I think we're going
to be.
(50:48):
We're going to kind of grow andevolve the industry.
We're going to help advance thetradecraft of practices.
We're going to help...
promulgate the value-addproposition to C-suite
executives and others.
I have been a recipient ofmentorship.
I have helped mentor peoplebefore.
(51:09):
We all kind of mentor eachother in a lot of ways, and that
just helps us growcollectively.
So just be good to one anotherand just try and pay it forward
as best we can.
That's, I guess, my upliftingway to end this recording on a
Friday.
Pedro Kertzman (51:24):
Man, honestly, that resonates so much to me.
I really love that.
Probably that's the mostinteresting part.
One of the most interestingparts of having this podcast is
to see how the community cancome together like random
strangers just to help eachother share knowledge with the
(51:47):
broader CTI or cybersecuritycommunity about the advantages
of having a CTI program.
And that's just so amazing.
I love this attitude.
And I think when you pick thisspecific quote unquote topic, As
a final thought, I think justgoes to show that it's really
(52:07):
something important in thecommunity.
John, thank you so very muchfor coming to the show.
I really appreciate all theinsights and I really hope I'll
see you around.
John Doyle (52:18):
Yeah, absolutely.
Thank you again for having me.
This was a ton of fun.
Rachael Tyrell (52:23):
And that's a wrap.
Thanks for tuning in.
If you found this episodevaluable, don't forget to
subscribe, share, and leave areview.
Got thoughts or questions?
Connect with us on our LinkedInWe'd love to hear from you.
If you know anyone with CTIexpertise that would like to be
interviewed in the show, justlet us know.
Until next time, stay sharp andstay secure.
Everybody there is just the smartest person in the
world and super humble.
Rachael Tyrell (00:04):
Hello and welcome to Episode 7, Season 1
of your Cyber ThreatIntelligence Podcast.
Whether you're a seasoned CTIexpert, a cybersecurity
professional, or simply curiousabout the digital battlefield,
our expert guests and hosts willbreak down complex topics into
actionable insights.
On this episode of Season 1,our host Pedro Kurtzman will
chat with John Doyle, who hasover 16 years of experience
(00:26):
working in CTI, digitalforensics, cyber policy, and
security.
Pedro Kertzman (00:57):
John, Thanks so much for joining the show.
It's great to have you here.
John Doyle (01:00):
Hey, Pedro.
Thanks for having me.
I'm really excited about this.
Pedro Kertzman (01:02):
Awesome.
Usually, I start asking theguests about their journey into
CTI.
Would you mind walking usthrough that,
John Doyle (01:11):
Yeah, for sure.
My story, I feel like, iscomical, much like a lot of my
colleagues in this field, how wealmost got here by accident.
My journey kind of starts atuniversity.
I was working...
2008 timeframe for our GeekSquad, or the equivalent of our
Geek Squad on campus.
(01:31):
And at that point, I had spenta few years, and that really
kind of got me into computers alittle more than I had
previously.
So there was that dynamic.
I ultimately went to gradschool for information security
and also picked up this degreein national security studies,
not realizing that kind of wouldposition me well for
(01:52):
understanding the geopoliticaloverlay of events that happen
and how there's cyber actionsthat are interrelated.
Spent a decade thereafterworking for the CIA as a threat
intelligence analyst focusedspecifically on cyber threats.
And then I eventually made myway to the private sector.
(02:14):
And in the private sector, Iwas working and working for
Mandiant and I also moonlight asa SANS instructor for the
threat intelligence course.
So I've kind of I've kind ofgone an interesting route, one
that I feel would resonate a lotwith people who used to work
for government, who kind of cameout and transitioned into this
(02:35):
field, and one that I think isbecoming a little more
non-traditional in the pastdecade or so, as the really
juicy data for threatintelligence analysts no longer
lives in government spaces andclassified sources, but actually
lives in victim environmentsthat incident response firms and
(02:56):
managed defense providers orMSPs actually have access to.
Pedro Kertzman (03:00):
That's interesting.
I always thought that stillNowadays, military intel
experience would be superdesirable, for example, on the
CTI field.
But it's interesting to knowthis shift.
I never thought about that, butit makes sense.
John Doyle (03:19):
Well, there's two parts to it.
I think you hit on aninteresting first bit there when
you talked about the skills.
So one of the things you get inmilitary intelligence or even
civilian and law enforcementintelligence is kind of grounded
tradecraft.
So you've got to be able tocommunicate.
communicate well, you got to beable to do research well that
can be thoroughly backed upbased on this or that data
(03:40):
source.
So you learn kind of the basicsof research and it gets
reinforced and then you learnhow to communicate it over time
to stakeholders.
So you're learning analyticalprocesses, which is incredibly
helpful irrespective of what jobyou have.
But the cyber element is what'skind of or at least the sources
(04:01):
of information is where we'veseen kind of a change happen in
the last decade.
Pedro Kertzman (04:06):
there you go yeah the other day i was talking
to somebody that they mentioneduh something like that uh the
intelligence knowledge shouldcome first than the cyber
knowledge you can combine thecyber into intelligence but if
you don't know how to dointelligence and you just have
the cyber knowledge then youkind of need to learn from the
get go a new trade craft becauseintelligence again even though
(04:29):
we leverage cyber security uh ifyou just do for example,
malware analysis, but then youneed to learn how to connect the
dots into a moreintelligence-driven type of
approach.
Does that make sense?
John Doyle (04:43):
Yeah, I can see that.
I can most certainly see that.
I think, I mean, if we back upa little bit and look at almost
the start of CTI, you had twodifferent audience types and two
different skill sets that wewere trying to speak to.
So you had the incidentresponders and the SOC analysts
(05:04):
doing kind of threatintelligence organically to make
a determination as to like, isthis bad?
Should I take an action on it?
Does it require immediateattention?
Can I build out a playbook?
But that's different thanspeaking to like an executive
level audience where they'relooking at a more strategic
overlay.
They're trying to identifycomplexities and trends in cyber
(05:28):
threats and how this relates totheir organization's risk
posture.
And we're actually seeing,definitely seeing kind of more
alignment happen and moreconvergence happen over the last
decade.
But it's just two differentskill sets.
I mean, on the former, you'vegot kind of a digital forensics
(05:48):
incident response background,cybersecurity background, moving
into a more higher levelservice support construct.
And from the other side, youhave traditional intelligence
analysts who are used toproviding insights in a way that
resonates with differentaudiences at a high level so
they can make informed decisionswho are then having to learn
(06:11):
the more technical and the morenuanced aspects of this field.
Pedro Kertzman (06:15):
Got it.
John, awesome.
And so you mentioned a fewaspects, important, very
important functions within a CTIteam as a whole.
Any other changes that you'veseen in the past possibly few
years within how those roles areevolving or changing based on
(06:39):
either the maturity of theindustry itself or from external
factors, for example?
John Doyle (06:47):
Man, you asked a loaded question there.
Yeah.
Let me think about this.
Yep.
We've seen...
a few different evolutions.
So there are more companiesthat have access to firsthand
information and also secondhandinformation that is really
(07:08):
important for building out anunderstanding of adversaries or
threat activity groups.
So on the vendor side, there'smore vendors with useful data
that exists that is happeningconcurrent to a high level
demand for threat intelligenceto help inform decisions whether
(07:31):
it's at the board level theexecutive level the cyber
security program manager levelor even the threat intel manager
level then there is the skillsso the demand from other
internal stakeholders is thenpredicating the need for
individuals on the teams to bebroader and deeper in their
(07:58):
specific craft.
And that's causing, dependingon the organization, the
resource, the budget, all ofthose kind of administrative
constraints and kind of growthprojections, causing individuals
to have some level ofspecialization, but also having
the need to have a baselineskillset or at least a
(08:19):
functional understanding of howwhat i do compares to what you
do and how it's complementaryand how we can then use that
together to work effectively todrive better outcomes
Pedro Kertzman (08:32):
got it okay that's uh that is interesting
you mentioned like part of theEvolution now, the way I
interpret it, it's bringing morevalue to all levels of the
organization and not anymorejust the SOC or just that
particular department.
(08:53):
Actually, I was provokingpeople the other day that threat
intelligence is not just threatfeeds and reports anymore.
You can have way more emissiondecision making.
You can use that CTIinformation to do way more than
(09:15):
just playbooks with feeds andunderstanding what's going on
with the reports and things likethat.
So it makes sense.
It's interesting and it's goodto see the industry overall
shifting in that direction.
One of the struggles I stillhear a lot from CTI folks is
(09:36):
being able to prove the value oftheir department into the upper
management levels.
So they don't get as impactedor sometimes with layoffs and
stuff like that, but even, hey,I can help you with this.
If you have like a decision todo, I can bring Intel and maybe
(09:56):
we'll quote unquote, enrich thedata that you have to take that
decision.
Sorry, go ahead.
John Doyle (10:09):
No, no, no, no.
I mean, you're hitting on twothings I think a lot about.
One, what's the best valueproposition for an internal CTI
team?
And then what is the valueproposition for intelligence
vendors who may service internalCTI teams?
And this is something that wheni'm working with clients who
(10:33):
are building their cti programsor trying to mature it these are
all decisions that they'retrying to make these are all
complex situations they'retrying to navigate is how do i
convince the business that wehave the ability to provide
proper level support to servicethese needs that are measurable
(10:54):
that actually show impact bothat the bottom line of the
company, but also showsefficiency gains.
So it's...
My boss at one point in timesaid something to me, and I
don't think that he meant it aslike this wise sage advice that
was super impactful, but itactually was.
He's like, look, look, John,like organizations use CTI for
(11:17):
like one of three reasons.
One is you've saved the companymoney and money could translate
to like brand reputation lossor a whole host of other kind of
things that are associated withrisk.
It's making the company moneyor you're improving efficiency
in some way, shape or form.
I was like, damn, that wasgood.
I need to, I need to take morementorship from you.
Pedro Kertzman (11:42):
Yeah.
Mentors.
Right.
So, so valuable.
I feel you.
Yeah.
Some quotes just have thatpower to stuck on our heads and
we will keep thinking about themand using as a guideline to do,
you name extra work research,something like that.
John Doyle (12:00):
There is like one more thing here.
Have you heard of the CTI-CMMbefore, the CTI Capability
Maturity Model framework thatexists?
Pedro Kertzman (12:11):
I heard of it, haven't had the chance to deep
dive into it yet.
John Doyle (12:18):
Okay, I'm going to give it like a 30 second plug
here.
So it's a model that's designedto effectively demonstrate that
value prop across differentstakeholders.
So here is incident response,here's SOC, here's hunt, here's
red team, here's third partymanagement, et cetera.
What is there?
organizational function do andhow can CTI support it?
(12:43):
And then it breaks it down overa spectrum of maturity levels.
What they just done in the pastmonth is actually created
metrics.
So I'm an internal CTI team.
What are good metrics for me tomeasure against if I am
supporting incident response, ifI am supporting purple team, if
I am supporting red teamexclusively to help with,
(13:06):
because it's hard, right?
Metrics, creation, is notsomething that like me as a
deeper practitioner have beentrained to do, or even like
somebody coming into amanagement role.
They've not gone to, you know,university or taken professional
training on like, how do Icreate effective metrics?
But my boss is asking me tocreate something, so I have to
create something.
(13:26):
so this framework is actuallydesigned to bridge a lot of
these gaps to help with thatvalue prop for intelligence
teams as a whole so like if ididn't have that organically i
don't have to create it fromscratch it now exists it's
something that i can take andpull from
Pedro Kertzman (13:41):
that's amazing that's that's super important to
how to to sell the value ofyour of your cti team uh and
maybe um i don't knowbrainstorming moment um do you
think cti could go uh and offeras well value to um some
(14:03):
companies used hr to manage uhcyber security awareness program
or you know work handy handwith it for that but do you
think for this type of programcti could also offer some value
John Doyle (14:22):
absolutely I think the closest partner that a CTI
team could have besides incidentresponse is security awareness.
And that's for a host ofdifferent reasons.
So one of the countries I usedto track a lot was North Korean
cyber threats.
So all of the groups associatedwith that.
(14:43):
And in the last year, One ofthe trends that we started to
see emerge, and it's not netnew, but it's kind of net new in
the public's eye, is thisnotion of North Korean IT
workers applying for roles atbespoke companies or even big
name companies, which hasfundamentally done something
(15:06):
quite unique, I think, in thisfield.
It's allowed us to not justlook at threat activity.
In part, some of these actorsare using their access for
threat activity.
or providing the accessremotely to cyber operators.
But it now lets us actuallybridge the gap because this is
something that HR needs to knowabout to be able to screen out
(15:28):
North Korean IT workers that aretrying to work for their
company.
It brings in the notion ofinsider threat and it really
kind of expands the aperture toshow what that value prop could
be for threat intelligence tosecurity awareness training, HR.
identity and access managementand like a few others and it's
that's not the only case toowe've been seeing but we've also
(15:50):
seen like i'll admit i was abit of a naysayer when it came
to you know using the dark webto surface things that are
useful for cyber threat researchBut like the notion of identity
intelligence is yet anotherarea that's starting to prop up.
So leaked creds, customer PII,internal PII that's leaked, like
(16:11):
all of that's really important.
And I think this just speaks tothe evolution of the industry
in the evolution and almostremit in things that are
expected of a CTI team to beable to cover, let alone
knowledge of like exploitationof edge devices and some of
these other things that weremore like, whoa, that's very
(16:33):
unique, specialized knowledge,but now it's almost expected.
Pedro Kertzman (16:37):
Yeah, no, that's great to, let's say, brainstorm
that with you.
I'm going around some localconferences here to try to give
real use cases andthought-provoking things to
people that CTI should go beyondjust threat feeds and reports.
(16:58):
you're basically short sellingyour value if you're just
focusing on that, right?
So it's great to hear that I'mnot alone.
Yeah, no doubt about that.
That's awesome.
Man, and you mentioned aboutthe CTI maturity model as well.
(17:19):
Any other interestingframeworks that we've seen
around that perhaps noteverybody is using and they
should use more or explore more?
Anything around that?
N
John Doyle (17:32):
I'm a little biased for what's going to come out of
my mouth as the author of it,but I'm going to do my best to
give it as an objective of asell as possible.
Pedro Kertzman (17:43):
Fair
John Doyle (17:43):
enough.
So a few years ago, we had beengetting asked a lot of
questions on the Intelconsulting side about, well,
what's the right composition fora CTI team?
What are the right skills?
What are the right backgrounds?
Who should I hire?
How should I hire?
It's all sounding veryfamiliar, isn't it?
So I kind of went out there andI'm like, well, let's see what
NIST has to offer.
NIST is a standards body.
(18:04):
They do a pretty good jobacross the board at putting
useful information out there.
Let's see what they have.
And I came across NIST SP800-181, which is the NICE
framework, the NationalInstitute on Cybersecurity
Education.
Oh, cool.
Fantastic.
Let's drill down into this.
(18:24):
And as you start to see all ofthe different role profiles that
would be part of acybersecurity program, CTI gets
parsed out almost into three orfour different categories.
You've got a collectionsmanager, you've got a threat
warning analyst, you've got soand so forth.
And as I looked at this, I waslike, I'm just curious how much
(18:49):
government influence there washere.
And whether there was anybodyfrom private sector who actually
weighed in on this.
So, you know, I ended upconnecting with two individuals
who were leading the NIST NICEproject, and we had a very
cordial conversation.
It was a very goodconversation.
And they're like, yeah, youknow, you're right.
(19:12):
A lot of this wasgovernment-backed.
Like, we kind of leveraged themfor, like, what does right look
like?
I was like, cool.
I work with threat intelligenceteams that are, like, six
people max you mean to tell methat one of their six people is
just managing collections andlike that's all they're doing
now I get it if you're workingin like a several hundred person
(19:35):
intelligence community thatthese are distinct roles but you
mean to tell me that there's anexpectation being made here
that like one person is doingjust this job exclusively like
yeah okay to each their own umSo, so really kind of prompted
me to come back to the table andI asked her, I was like, well,
(19:56):
I've got a lot of ideas herefrom what I'm seeing.
Is there a way we might be ableto like merge this from like
your knowledge and skills andabilities you've broken out or
your KSAs you've broken outacross the different, um, the
different role profiles.
And they're like, yeah, butwe're actually thinking we might
switch the model up a littlebit and it's going to actually
potentially break things.
(20:16):
So like, if you want to go outand do something on your own and
create your own framework,that'd be, that'd be cool.
Like we'd love to collaborateor at least have insights.
I was like, yeah, I could dothat.
So this led to the creation ofthe Mandiant CTI Analyst Core
Competencies Framework, which isbroken out into four different
pillars.
(20:36):
Those pillars functionally canbe broken into two categories.
The first category is softskills and professional
effectiveness.
And the second category istechnical acumen and threat
knowledge.
So we enumerate 182 differentknowledge and skills and
abilities inside of thisframework itself, where if
(20:58):
you're interested in justjoining the field or like you've
been in it for a while andyou're like, all right, what's
next?
You could actually rateyourself.
So there was kind of threedifferent cruxes for why we
created it.
The first was self-inventoryevaluation for personal
development and professionaldevelopment.
Second was to helporganizations with hiring
decisions because it's like,man, what do they need to know?
(21:20):
Like, I feel like this guy justneeds to know more than the
MITRE ATT&CK framework and whatit's used for.
Like what are those otherthings?
So it was designed to alloworganizations to almost lift the
respective KSAs to include intheir job requirements.
And then we had a third onetoo, which I kind of bled into a
little bit already with thedescription of the other two,
(21:41):
but it's for that evaluation.
How do I evaluate where mystaff are and how they can grow?
So creating almost like a teamreport card across the different
areas based on the roleprofiles.
So that's one that I think Ithink it works quite well.
It carries the Mandiant namebecause I was being paid while I
(22:01):
developed it, but it was donein coordination with the private
sector.
A bunch of my peers who are inthe industry here, either at
vendors or working simply inprivate sector and also in
public sector too, they wereable to weigh in on it.
So that's, I think, one of theframeworks that maybe got me
invited to work on this cti cmmproject okay um so it's it's
(22:28):
been good i mean the the thegenesis of a lot of these
projects is to help fill gaps inindustry whether it's showing
up knowledge or helping programsbuild to better because we're
all in this together right it'sa small industry
Pedro Kertzman (22:42):
man well yeah that's great let me uh digest a
little bit here and pause for asecond.
I like it.
And by the way, just mentioningabout the Sense CTI conference,
it reminded me, any other goodCTI-related conferences you had
(23:05):
the chance to attend?
Any values you saw on this orthat other one?
Anything to comment on that?
John Doyle (23:15):
I love this.
I've been thinking a lot aboutthis too.
Yeah, there's a whole handfulof them and I think it speaks to
kind of both the specializationand the growth of the industry
in different geographies.
Some conferences are bettersuited for generalist audiences
(23:36):
or those who are kind of entrantinto the field, maybe upwards
of three, four years.
Some are better for researcherswho have been around for a
little while longer looking toconnect.
And conferences usually takeone of two forms.
One, it's open and you pay toget in, or sometimes it's free
for live streaming.
The other is kind of like thisclosed trust network where you
(23:57):
need somebody to vet and verifywho you are and that you're
trustworthy because informationbeing shared might be shared at
a certain sensitivity level likeTLP Red.
Only the people there can talkabout it.
Maybe we start with the SANSCTI conference since you already
mentioned it.
That one happens in January.
It's in DC.
(24:18):
DC, Alexandria, Arlington area,usually.
It's been a good one.
It's been one of the longerstanding ones that exists.
Free to stream, nominal fee toshow up there in person, usually
maybe about 250, 300 show up inperson.
It's a really good way tonetwork.
And maybe I back up for asecond.
(24:39):
There's different valuepropositions for people who go
to conferences at differentlevels.
And it's a little weird too,because I feel like a lot of us
on this field, we tend to beintroverts.
So like our social battery isdrained and like going up to
somebody like thought of me andlike, hi, I'm John, who are you?
And what do you do?
And who do you work for?
And like the 20 questions, it'sjust intimidating.
Pedro Kertzman (25:03):
I'm happy to know it's not me.
It's only not me.
Oh man.
I go back to the hotel.
I'm like crashing, crashing.
John Doyle (25:10):
Yes.
Yeah.
No, but like, that's, that'snormal.
And like, that's, it's funny.
Cause my junior analyst, Italked to a lot and I'm like,
you just gotta put yourself outthere.
Like at some point, everybodylike we have this shared frame
of reference where it's likeshared experiences so like if
you're being a little sociallyawkward guess what a lot of us
are socially awkward so it'sfine but we're in our own heads
(25:31):
most of the time where it's likei don't know but what if i said
what if i sound silly and i saysomething that i shouldn't like
whatever
Pedro Kertzman (25:37):
everybody
John Doyle (25:38):
like we're our own worst critics so it's fine yeah
So the SAN CTI conference isreally good for actually
learning.
They actually did a split tracklast year and the year before
where they had kind of like newto cyber.
So people who are kind ofgenerally in that first three,
four year bucket.
And then they had like a notlike they didn't call it
(26:00):
anything special, just like thisis like the other track that we
have.
It's kind of cool because I sawa lot of individuals who were
in person who had been seasonedpractitioners for like a decade
plus stop in for some of the newto cyber talks and they have a
whole Slack channel where peoplecan ask questions and bounce
(26:21):
ideas off of each other.
Like I saw some of the moreseasoned analysts and
researchers actually answeringquestions that were being asked
during the presentation in theSlack channel.
So it's like a way to kind ofhelp mentor and build people up.
So I'm actually a big fan ofany conference that has a
Discord server or a Slackchannel that's already kind of
(26:41):
pre-established to allow forthat kind of growth amongst
peers.
So that's, that's San CTI.
There's like a few that arecoming up soon.
There's a cyber crime onecalled sleuth con that's coming
up in June.
It's not the only cyber crimeor underground type economy one.
So like team, team comry rise,it's in three different
(27:03):
locations to cater to different,uh, analysts in different
geographies.
One is in, um, Singapore forthe APGA region.
One is in Europe and one is in,I believe California.
And they happen, um, you know,at different, times throughout
the year.
Those are good ones.
You've got SleuthCon, which iscyber crime focused.
I talked about that.
(27:24):
The EPT or nation state versionof that is CyberWarCon, which
takes place in Washington, D.C.
in the fall.
But there's more.
There's a whole host of otherones.
So for the EU segment or eventhe UK segment, I'll kind of
include them as adjacent andcombined for this.
You've got cyber threats, whichthe UK NCSC puts on.
(27:46):
You've got Virus Bolton, whichis actually taking place in just
a few weeks.
You've got First CTI, which isusually hosted in either Berlin
or Munich.
Then you've got TIX, which isout of the Netherlands, the
Threat Intelligence Exchange.
And I feel like I'm missing oneor two there, but I feel like I
(28:06):
hit on most of the high-levelones.
Oh, and then there's some ofthe private closed ones, which
include like LabsCon andPivotCon.
And Woo, and maybe a fewothers.
Pedro Kertzman (28:20):
Cool.
And any of those conferences...
Would you say it's like amust-go for seasoned CTI guys?
Like a must-go if they don'tknow yet?
John Doyle (28:31):
Yeah.
Cyberwarcon and Sleuthcon...
Sleuthcon's kind of new.
It's only been around for a fewyears.
It used to be Brunchcon,because it was the day after
Sleuthcon, but then it got kindof parsed out as its own thing.
You end up meeting...
a lot of really interestingpeople at both of them who have
(28:52):
been practitioners in the fieldfor a very long time.
And I mean, one of the thingsthat resonated with me at both
of those conferences, and maybethis is more representative
reflection on the field, is likeeverybody there is just like
the smartest person in the worldand super humble.
Everyone's willing to talk toyou about anything and help kind
(29:14):
of mentor and grow.
So the networking dynamic youget from either of those
conferences is wild.
Pedro Kertzman (29:21):
That's so nice.
John Doyle (29:21):
But of course, the drawback is you're in the U.S.
around Washington, D.C.
So for some people, that's costprohibitive if they're trying
to travel internationally, whichis why I really like the
evolution and advancement ofthese kind of EU-type equivalent
conferences to make it moreaccessible, this field to be
more accessible.
(29:42):
Because you still get someresearchers who speak at the
conferences who come from theU.S.
or will come from EU or willfly in.
So it's not like there'sdiminished value in any of them.
It's just they're kind ofsometimes different flavors of
the quality you get.
But the people who go, topnotch across the board.
Pedro Kertzman (30:06):
Yeah, that's a great point.
I think the audience willchange, but the quality of the
speakers might be similar,especially for the conferences
paying for flights and stays andall that.
So, you know, if people want togo there and have their session
or keynotes, you know, it'seasier for them.
But it's not as, like youmentioned, prohibitive for
(30:27):
people.
thousands or hundreds of peopleflying from one continent to to
the other yeah i think blackeven black hat now has like a
apj uh europe and and us ofcourse
John Doyle (30:41):
yeah and if you're looking for kind of more local
cyber security conferences so ifwe extract a little bit outside
of cti the b-sides conferencesare great to go to
Pedro Kertzman (30:49):
oh i love
John Doyle (30:50):
them there's local chapters of these sites just I
mean, throw a rock in any whichdirection, you're going to find
a local chapter.
Pedro Kertzman (30:58):
Oh, yeah.
Honestly, I think any mid-sizedcity in North America, I would
say, probably has a B-Sides atthis point.
Or a metropolitan area.
Yeah, yeah.
Awesome.
Yeah, so changing gears a bit,you mentioned you used to work
for the CIA.
(31:18):
Of course...
you know, details arecompletely confidential, but
generically speaking, anyspecific part of the job, how
used to be like any part of theroutine you could mention and,
uh, any insights from that, um,and, and maybe from a broad
audience perspective, thingsthat could, uh, people could
(31:42):
leverage from the overall, uh,industry.
Unknown (31:46):
Yeah.
John Doyle (31:46):
Yeah, no, that's fair.
I'm not sure that I have anythat I can publicly share, but
there's a lot of things thathappen behind the scenes that do
make their way into the news.
And likewise, the news cyclehelps prompt some investigations
into different actor activitiesfor analysts or analysts.
(32:07):
So we're always looking forthings that are related to
whatever our focus is.
So open source being one ofthem was absolutely something
that I was looking at.
And I was like, oh, this groupis saying this.
And then for us, maybe therewould be a policymaker in
Washington, D.C.
who would read this particularnews thing and ask a question
(32:28):
like, hey, what does this mean?
Should we care about it?
And then the question wouldfilter its way down to the
respective organizations andwe'd answer it.
We'd have the opportunity tokind of weigh in, add some
ancillary information about it,maybe the actors, maybe the
groups, maybe the types ofactivity, and not just help them
(32:50):
with the immediate ask, butalso anticipate what other
information they might need.
And this relates to this trendthat we're seeing, which then
allows for some opportunityanalysis on top of that to help
make life harder on threatactors, or clamp down on threat
activities, or help guide youknow, The funding streams of
(33:16):
different initiatives or justotherwise kind of highlight gaps
and areas where the policycommunity could plug in to
really help fill them.
So in a lot of ways, it waskind of a really cool
environment that I found myselfin that I didn't think a defer
practitioner had that type of aflair or could have that type of
(33:38):
a national level impact.
But yeah, kind of here we are.
It is unfortunate that theagency is very close-held about
some of their successes thatcome out because I think it'd be
really cool for some of thestories if they did make their
way public.
Pedro Kertzman (33:54):
Yeah, share best practices.
I would imagine things of thatnature, right?
Yeah, it's a complicatedtrade-off, right?
Because you don't want toexpose those best practices
because people can, on the otherhand, could leverage that to
maybe bypass them.
But it's interesting to thinkthe line, right?
(34:21):
Where's the risk of sharingthat, but also the risk of not
sharing that with a broaderaudience that could, I don't
know, become targets, forexample.
So it's a tough decision tomake, I would imagine.
John Doyle (34:37):
Yeah, so on the one hand, the stories about what was
worked and what the results forthe impact might not come out.
But the tradecraft, I think, iscoming out for threat research
because we're seeing, and wehave seen for the last decade
(34:59):
plus, a lot of really talentedindividuals leave government to
go work for private sectorvendors or individual companies.
Then you've got the ISACs orother trusted communities or
these conferences where it'sopportunities to share best
practices in how we go aboutdoing tracking, how we do
(35:22):
alignment to stakeholders, howwe find signals in the noise
that are interesting and buildthat intuition.
There's I feel like a lot ofthat is out there.
A lot of it's becoming morepublic knowledge, certainly a
lot more than like a decade agoor more, because of the high
(35:45):
prevalence of people who haverotated out from government
space, military intelligence,civilian intelligence, law
enforcement.
And honestly, I think that'sprobably done in insurmountable
amount of good for buildingresilience for building cyber
(36:05):
security for a bunch oforganizations that would
otherwise be victimized um thatat the time was probably
scrutinized like you're leavingus to not work on the mission
you've lost like you you've lostyour your vision you've lost
your perspective it's like nahnot really i'm still getting
mission impact so Overall, Ithink what was a bad news story
(36:29):
for governments with theattrition turnover actually
turned out to be a really goodthing for bolstering this
industry and bolsteringcybersecurity resilience for
organizations to help thembounce back and detect
ransomware to help.
to help foster public-privatesharing, to help grow the
(36:52):
overall security posture at thenational and at the economic
level, right?
So, I don't know.
You got me on a rant there.
Pedro Kertzman (36:59):
No, this is actually an excellent
perspective.
Thanks, John.
I appreciate it.
It makes total sense.
I think at the end of the day,everybody is benefiting from
that perspective.
pool of amazing resources thatthe government in some shape or
form was able to provide andtrain.
(37:20):
You know, like you mentionedbefore, right?
This is a team sport.
We can only win this together.
If you have like a superknowledgeable, insane expertise,
part of the equation let's saythe government from municipal
(37:44):
states uh federal level but thenthe companies you know are
falling apart from a cybersecurity standpoint then you
start getting problems with theinfrastructure manufacturing and
then as a whole it's just notgoing to be positive to to
anybody right anyhow um Youmentioned Isaac, and maybe
(38:12):
piggybacking a little bit on oneof our previous topics, do you
see the Isaac role,quote-unquote role, is also
evolving?
Some of them, or are they...
primarily focused on sharingthe more traditional forms of
(38:34):
CTR, like feeds, reports, or yousee the role of the ISACs also
changing over time?
John Doyle (38:45):
That's a tough one.
So when we say ISACs, what wemean is information sharing and
analysis centers, ISAC being theacronym for it.
Every industry kind of hastheir own ISAC or ISO.
For our purposes, let's justuse the word ISAC to avoid
technical distinctions.
Every industry owns andoperates the ISAC model just a
(39:08):
little bit differently.
So the standards and practicesin play The membership terms and
service for the healthcare ISACversus the retail ISAC versus
the financial ISAC versus the ITISAC, you know, so on and so
forth, are all going to vary.
So you'll have differentstandards levels for each.
(39:29):
I'm seeing, so I work a lotwith the Healthcare ISAC and a
few others that I won't name, toreally kind of help bolster
their capabilities.
And you're right, historicallyit's been, well, maybe we just
share out a bunch of IOCs.
Some of these ISACs areactually looking as part of
(39:52):
their membership criteria forthe participants, the industry
partners that are operating inthat sector to share with them
threat insights to then shareout at a sensitivity level.
So whether that's TLP Amber orTLP Amber Strict that says us
members here, acrossorganizations, we can use it for
(40:16):
the intent purpose of huntingthis type of adversary activity,
or because we're trying todetermine whether or not this is
a campaign.
So something has come in, atipper has come in from an
industry partner, and they say,hey, we're seeing this
proactively.
Is anybody else also seeingthis type of activity?
So in a lot of ways, the ISACsend up being a nice early
(40:38):
warning system to determinewhether there's a campaign of
significance, and chances arethat that information is
probably also being shared inother closed channels too.
But there's this push and pullmodel.
This push and pull model, andI'd actually go out on a limb
and say that we are seeing anevolution over time for getting
(41:01):
vendors and other researchers inthere to help brief the ISACs
at like their annual orsemi-annual conferences up on
tradecraft.
Or what does best practiceslook like?
But that's going to be...
ISAC to ISAC, so I can't make ageneral statement there because
of the way they're owned andrun.
(41:23):
But I have been pleasantlysurprised in the last few years
with the ISACs that I've beenworking with to see them grow,
to see them take on board andtry new things or have a
centralized threat intelligenceplatform that can act as a push
and pull model that has reportsthat are provided with those
different classificationtaglines to allow them to be
(41:46):
able to share things back andforth
Pedro Kertzman (41:48):
awesome and and john one of the things we hear
probably every single day nowmany times it's uh ai uh you see
any um kind of good usage orshifting uh across our industry
if you will uh when it comes tothe usage or where ai could help
(42:12):
us uh um on any directions youcan think, from a detection
standpoint, LLMs, you name it,any sort of AI over there
helping the CTI industry?
John Doyle (42:28):
Yeah, maybe I'll start with a quick story.
So I was asked this about ayear and a half ago by some
colleagues.
We were brainstorming, and thenagain by a client.
And then at a workshop, I kindof put on my slide that says,
AI.
What is it good for?
And then I just put up a bunchof different high resolution
(42:51):
fantasy images.
And I go, it's great forcreating characters for your
Dungeons and Dragons campaigns.
And it got a good laugh.
But it's true.
There's a lot of like-minded orlike culture type of like nerd
culture here where we've all gotour own kind of hobbies and
things.
And image generation isabsolutely one of those.
Go to find out with things likeMid Journey and Claude and
(43:14):
other kind of models there.
But as far as CTI specificapplications, applications.
If you look at the intelligencelifecycle, all five phases of
it, there is application for AIin all of them.
And when I say AI, I'mspecifically honing in on AI is
a tool that's used by humans.
(43:34):
AI is designed to augment ourcapacity and our capabilities.
So it is something that's usednot by itself necessarily, but
to help us with things like datatriage, like looking at large
scale data sets or leaks of dataand being able to surface
insights pretty quickly, a lotquicker than the human would be
(43:57):
able to.
It helps us with standardizingreporting.
I use AI almost every day forwriting things like, actually, I
just used it yesterday.
I said, hey, I need to write apair bonus for this guy.
He did the thing.
Please include this, this, andthat, and put it in a formal
tone that highlights thesedifferent attributes that I know
(44:19):
are going to be useful forhelping maybe get him promoted.
It spits something out for mein about 30 seconds that
otherwise would have taken me 15minutes to write.
And even if it gives you the80%, it's a really good starting
point.
When you start layering on topof that kind of rag, the
(44:40):
augmentation for it to gooutside of its closed data
sources and go doing deepresearch, it really kind of
shows a value add prop.
Now, kind of the drawback withthat is taking anything that the
AI gives you as truth withoutdoing vetting and validation.
So for us as analysts, I feellike a A lot of more senior
(45:02):
analysts are using this almostlike a search engine, like on
steroids.
And a lot of the more junioranalysts are just taking it and
saying, all right, this isgreat, we're done.
So like that critical thinkingand that trust but verify
mentality is quite important.
But as far as producinginformation in a quick way,
doing triage, outputtinginformation in a standard
(45:22):
structure, like please extractall of the IOCs from this
particular report and map themto MITRE ATT&CK.
it does that pretty quickly.
Oh, and please put that in atabular format so that I can
take this table and copy andpaste it into like a spreadsheet
or give me this in a CSV file.
It's really good at doing taskslike that.
(45:44):
Things that otherwise wouldhave taken us a lot of time to
do the manual curation of andtransformation.
It can do that pretty quickly.
Speaker 01 (45:51):
I've
John Doyle (45:52):
used it a little bit for the vibes coding scripting.
It works fine enough.
I'm just right.
I'm not a developer.
I can write a few scripts.
It can write the scripts inPython a lot better than I can
though.
Anytime I need to do a parsingexercise or I need to link
datasets together, I always justtake it at face value and
evaluate like, is there anythinghere that I need to change?
(46:16):
I will review the code base,but ultimately it's still the
human in the loop.
I don't know that we're gonnalose the human in the loop, but
boy, are we certainly becoming alot more productive as a result
of it.
So it's cool because in a lotof ways, AI is letting those
resource constraint CTI teams oflike two or three people really
operate at the level of like afive or six person team.
(46:37):
If it's used right, if it'sallowed in the environment based
on the risks that have to beaccepted for it, then there's a
whole host of otherconsiderations too.
But at the same point, it'shelping kind of bridge the gap
and allowing us to support thesedifferent stakeholders in a way
that historically we wereresource constrained and
couldn't.
Whereas today, it is reallykind of transforming the way we
(47:04):
do intelligence.
as a technology enabler but notas a replacement if that makes
sense
Pedro Kertzman (47:14):
100% I agree with that it brings me to a
point whenever we're trying tolet's say learn more about CTI
and I'm not necessarily talkingabout the feeds the latest
threats IOCs and reports and soon You have any go-to source to
(47:40):
learn more about the industry ingeneral outside of conferences
or new frameworks out there,things related more on the CTI
holistic approach, any go-tosources for it?
John Doyle (47:59):
Katie Nichols and Andy Piazza, both independent of
one another, pulled together toblog posts on their medium
sites.
On their medium sites, it'ssomething like the Newcomer's
Guide to CTI, really designed toprovide a lot of resources on,
well, what does seminal thingslook like?
(48:20):
What are some of these keyresources to understand and
really get into this field?
Less so on what CTI does, butthat's implicit and kind of
covered indirectly as part ofthat.
There's also...
the SAN CTI Annual Survey.
Are you familiar with that one?
No, I don't think so.
So it comes out every year.
(48:42):
They've been doing it now for ahandful of years.
It really gives goodperspective not on the vendors,
but on where industry trends arecoming and going.
So it is volunteer by nature.
They usually put like a two,two and a half month call out to
get data for people to take thesurvey to fill in, you know,
(49:04):
exact thing you would expectfrom it.
Like what industry are you in?
Like what's your, you know,average years in or whatever.
And like, capturing some kindof meta statistics that could be
used for vetting.
This year's one, I think, isdue to come out in about a
month.
I would be surprised if wedidn't see the inclusion of AI
in it, if we didn't see theinclusion of these different
(49:27):
like intelligence sources andvendors coming out.
So like the identityintelligence, for instance, like
that sounds like something thatI've been seeing kind of the,
you know, Intel 471s, theflashpoints and others really
kind of digging into dark webmore and having like the
recorded futures, like add adark web module or Palo Alto's
(49:47):
like add that as part of theirfeeds.
So like, I would be verysurprised if that, know wasn't a
trend that they tried tocapture in it too so like what
are those data sources beingused by internal cti teams
beyond like internal internaltelemetry um so it's it's really
good for kind of providing alay of the land and i think
(50:08):
that's what um that's where iwould go if i'm trying to get
kind of holistic capture atleast a quick snapshot of what
the industry looks like
Pedro Kertzman (50:16):
Oh, awesome.
Great information.
Thank you.
Make sure people listening thatyou check those medium sites
for more resources.
Excellent.
John, any final thoughts?
John Doyle (50:28):
Maybe we end on an uplifting note.
We're better together.
So this whole field, I feellike, is predicated on a bunch
of smart individuals who want todo the right thing, who want to
impose costs on theadversaries.
So the more opportunity we haveto work with each other and
grow from one another, thebetter off I think we're going
to be.
(50:48):
We're going to kind of grow andevolve the industry.
We're going to help advance thetradecraft of practices.
We're going to help...
promulgate the value-addproposition to C-suite
executives and others.
I have been a recipient ofmentorship.
I have helped mentor peoplebefore.
(51:09):
We all kind of mentor eachother in a lot of ways, and that
just helps us growcollectively.
So just be good to one anotherand just try and pay it forward
as best we can.
That's, I guess, my upliftingway to end this recording on a
Friday.
Pedro Kertzman (51:24):
Man, honestly, that resonates so much to me.
I really love that.
Probably that's the mostinteresting part.
One of the most interestingparts of having this podcast is
to see how the community cancome together like random
strangers just to help eachother share knowledge with the
(51:47):
broader CTI or cybersecuritycommunity about the advantages
of having a CTI program.
And that's just so amazing.
I love this attitude.
And I think when you pick thisspecific quote unquote topic, As
a final thought, I think justgoes to show that it's really
(52:07):
something important in thecommunity.
John, thank you so very muchfor coming to the show.
I really appreciate all theinsights and I really hope I'll
see you around.
John Doyle (52:18):
Yeah, absolutely.
Thank you again for having me.
This was a ton of fun.
Rachael Tyrell (52:23):
And that's a wrap.
Thanks for tuning in.
If you found this episodevaluable, don't forget to
subscribe, share, and leave areview.
Got thoughts or questions?
Connect with us on our LinkedInWe'd love to hear from you.
If you know anyone with CTIexpertise that would like to be
interviewed in the show, justlet us know.
Until next time, stay sharp andstay secure.
Popular Podcasts
NFL Daily with Gregg Rosenthal
Gregg Rosenthal and a rotating crew of elite NFL Media co-hosts, including Patrick Claybon, Colleen Wolfe, Steve Wyche, Nick Shook and Jourdan Rodrigue of The Athletic get you caught up daily on all the NFL news and analysis you need to be smarter and funnier than your friends.
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com