February 3, 2025 • 57 mins
This episode explores the intricate balance between career aspirations and parenthood, highlighting how remote work has transformed traditional workplace dynamics. The conversation touches on evolving priorities, the impact of AI on cybersecurity, and the challenges of pursuing advanced education while managing family responsibilities.
• The shifting nature of work-life balance for parents
• The importance of remote work flexibility
• Experiences in the cybersecurity field and investigations
• The role of AI in cybersecurity and privacy concerns
• The challenges of returning to education with family commitments
Follow the Podcast on Social Media!
Tesla Referral Code: https://ts.la/joseph675128
YouTube: https://www.youtube.com/@securityunfilteredpodcast
Instagram: https://www.instagram.com/secunfpodcast/
Twitter: https://twitter.com/SecUnfPodcast
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:01):
how's it going?
Keith, it's great to get you onthe podcast.
You know, I think we've beenplanning this thing for a while
and you know I got burnt outtowards the end of last year.
It just seems to happen everysingle year I get burnt out.
And then I went and, you know,decided to get sick with the
rest of my family for all of theholidays.
So I'm glad that we can finally, you know, get you on here and
(00:22):
have a great conversation.
Speaker 2 (00:28):
Thanks for having me.
I'm glad you're feeling bettertoo, yeah.
Speaker 1 (00:29):
It's a.
It was a blur last year too, Iagree with that, yeah, yeah, I
mean last year, I mean literally, you know it was December and I
mean my head was still in May,it was still like thinking about
the summer and stuff.
You know it's I don't know.
I think when you have kids,like everyone says that it like
speeds things up, you know, butlike it's so true, because every
single day is, you know, I havea two year old, right?
(00:50):
So every single day is herlearning a new word or
interacting with us a differentway, or, you know, giving us
attitude a new way, right, Likeit's all, it's all fun and new.
And when you have somethinglike that, I mean it just, it
just flies by.
Speaker 2 (01:03):
And when you have something like that.
I mean it just.
It just flies by Absolutely.
Mine are older, mine are likehigh school and college and my
one recommendation is going tobe don't blink, cause it's like
mine came home for um uh, youknow, the holiday break and he
just left and I think we're thenext time he comes home is going
to be close to Thanksgiving.
So it's like with a little man,just just adore that time.
Speaker 1 (01:25):
Yeah, you know.
So it's like, yeah, with alittle man, just just adore that
time.
Yeah, you know it's.
It's fascinating, right, how,how priorities change and
whatnot.
And I was always the kind of guythat when a when a better
opportunity would come along,like I would just take it.
I wouldn't even think about itor anything like that, right.
Well, now I've been remote forfive years, right, and I have a
two-year-old and I have a secondone on the way, due end of
(01:47):
April timeframe.
And fairly recently I hadsomeone offer me a job, better
opportunity, right, really bigfinancial institution worldwide,
a lot more money, right.
And I mean, five years ago Iwould have taken that without,
without thinking about it.
Right, the big problem was thatI'd have to go into the office
(02:10):
three days a week and I satthere and I thought about, like
the pay difference isn't worthme not being home when my kid
gets here, right, like it's thatit's a weird, it's a weird
predicament, because even 10, 15years ago, right, when I was
getting into the field, I meanit wasn't even, wasn't even an
option, right, like it was, likeyou're going into the office,
(02:31):
that's what it is, you know, ifyou can't live with it like get
out, get out of the industry,you know.
But now, like we have thatprivilege of being like, nah,
I'd rather, I'd rather be remote, to save the two, three hours
of commute, you know, just so Icould have those hours with my
kid.
Speaker 2 (02:49):
Yeah, the company I work for right now.
Their name is Coralite, andmost of us, so I work in
Coralite Labs, which is aresearch branch of Coralite.
We make the algorithms thatyou'll see on the sensors and we
have researchers that areliterally scattered all over the
world and it's really kind ofneat because me, being on the
(03:10):
east coast, I'll wake up at acertain time and I'll see a
certain set of people on butthey kind of fade off and then
midday a new set of people comeon and it's like we almost have
24 hour coverage, because evenfor a small team, because we're
scattered so much, and it makesit really beneficial for a lot
of things that we do, like keepthe processes running and doing
research around the clock andstuff like that.
Speaker 1 (03:27):
Yeah, it's way more beneficial and it makes so much
more sense for technology,people, people in IT, to be able
to be remote, right?
I mean, if the pandemic taughtus anything, it is that there's
literally no reason for us to bein the office, like literally
zero.
Right, you can't use the excusethat technology isn't there,
(03:49):
it's not available.
You know we don't have it.
You know the VPN only takes somany, so many users at a time,
right, like, we don't even havethat big of a trunk, right,
whatever those excuses were, allof that is completely gone.
You know them saying, oh, thebusiness won't survive.
It's like, well, you survivedfor two and a half, three years
during the pandemic witheveryone a hundred percent
(04:11):
remote.
What it like.
What are you doing now?
Right, like you know, Iregularly have an Amazon
recruiter reaching out to meevery single, every year, every
six months, something like thatRight Seeing if I want to come
on.
And this past year, every sixmonths, something like that
right Seeing if I want to comeon.
And this past year, or maybe youknow, four, four ish months ago
, one reached out to me and saidyou know, it was like two weeks
(04:33):
after the five day work week.
You know, push from Amazon,right when they announced it.
And I mean, he reached out tome and the very first thing I
said was, no, I don't think I'minterested because I wouldn't go
into the office five days aweek.
And first thing I said was no,I don't think I'm interested
because I wouldn't go into theoffice five days a week.
And that's really sayingsomething, because even a year
ago, I interviewed with a teamthere and they were three days
in the office and I asked him Iwas like well, where's everyone
(04:53):
located?
And the hiring manager was likeoh yeah, some people are in New
Jersey, new York, washington,seattle.
You'll be the only one inChicago.
And I said so I have to go in towork remote.
He said oh, you don't have tothink about it like that.
I was like no, that's what itis.
My entire team is around theglobe right In different time
(05:17):
zones.
I'm never going to see themface to face in person.
You just told me I won't, right?
So what's the point?
And he couldn't answer thatquestion.
Obviously, I ended up not goingthere.
But you know, it's like thesecompanies, they're really
pushing hard for people to goback in when, like you just said
, it's actually moreadvantageous for you to be
(05:39):
remote, because now you have24-7 coverage on a team that
otherwise would never have 24-7coverage.
And if a service provider evenclaimed that they could provide,
you know, coverage on threatresearch or security researching
, they're lying to you becausethat doesn't really exist.
Speaker 2 (05:56):
Yeah, one of the in our area, around the Washington
DC area, there's an interestinglittle twist because I, like you
, I see these things come across.
You know LinkedIn and stuff,and around here it tends to be
government contracting type ofwork and most of that type of
work tends to be classified.
So it's like if you seesomething that's I don't know
(06:17):
from like Raytheon or some youknow really big defense
contractor, you almost knowimmediately you got to go in for
that and do the classified work.
And you know, for some people,if they're not, they're not
living in DC, they're not goingto be able to do that.
Speaker 1 (06:30):
Yeah, yeah, no, that's that's a really good
point.
I mean, that's the.
I feel like that's probably oneof the very few industries as a
whole.
That it's you know.
It's forever in the office.
There's nothing you could doabout it.
And it's you know it's foreverin the office.
There's nothing you can doabout it, and it's you know
within within reason too right.
You wouldn't want someone takingtheir work home, like that's
illegal, yeah.
(06:51):
Then then it was started like awhole other conversation of
well, if it's legal now, thenwhy was Edward Snowden illegal
and you know all that, all thatsort of stuff, right.
But you know, keith, I reallywant to dive into your
background right.
How you got started, what madeyou want to get started right,
(07:23):
because you know I view securityas kind of like a crazy, not
necessarily wild west, but ofdive into security.
So how did you find your wayinto IT and how did that evolve
into security?
Speaker 2 (07:45):
for it but also kind of lucky.
So if we go back way back, Imean in the eighties, I can
remember programming on VIC-20sand Commodores and stuff like
that as a kid and I was justalways really, really interested
in computers and computers.
You know, if you're not as oldas me and you're watching,
listening to this, they were notall that common back then.
So if you had a VIC-20 aroundthat was a big deal as a kid.
So you know I got to learningprogramming and stuff like that
and then by the time I switchedto college there wasn't a
(08:08):
computer security anything inthe 90s, the early 90s, when I
went to college and when I didthat I knew I wanted to be in
computers and the easiest thingor the most matching major for
that was computer engineeringand electrical engineering.
So I did a double major forbachelor's in that and then by
the time I was done with thebachelor's I still kind of
(08:28):
looked around and I was like Idon't really want to design
chips the rest of my life likeCPUs and stuff like that.
I don't want to write operatingsystem code.
I know I'm interested in thisother stuff that during that
time I was working at the schoolof criminal justice too.
So I had exposure to cybercrime and just all sorts of
(08:49):
crime.
I even worked with them onetime help them clean up a blood
splatter room, so it was kind ofit was a really neat as a
computer guy.
It was a really neat job tohave back in the nineties.
So I got to see the lawenforcement side of side of
things which kind of went withcomputer crime.
But still nobody was calling itcomputer crime back then.
(09:11):
And so I graduated with mymaster's in 99 and I figured you
know what, I don't want to geta PhD yet because there's
nothing in here that's computercrime related.
If I did it it would have beenstraight electrical engineering
or computer science back in 99and it just didn't fit.
So I went to work and, um, Istarted out because I had a
(09:32):
programming background.
I helped work on some securitysoftware, some like log
correlation type of tools Ithink Splunk, but like way
before Splunk, just to you know,match web logs to a login and
not logins, but like yourfirewall logs and stuff like
that.
And in that I ended uptransitioning into doing
(09:52):
investigations.
So when I say investigations Imean very loosely, not just, you
know, remote attacker breakinginto your network but also your
internal employee gone rogue, orsometimes it was civil stuff
where one company would accuseanother company of stealing
their source code and I'd haveto look at it and say yes or no.
This is, you know, this is orisn't the same as what you think
(10:14):
it is, and I worked in thatspace for quite a while,
somewhere between like two earlytwo thousands through about 20,
16 ish.
So I did investigations forquite a while and in there I I
had experience doing experttestimony in federal court, um
(10:35):
criminal and civil.
I've done state courts liketennessee and some other states
that off the top of my head I'mnot remembering right now, but
you know a handful of otherstates and um it gave me
experience with a lot ofdifferent types of cases.
So some of the cases were,let's see, for the testimony.
A lot of the testimony tendedto be insider cases.
(10:56):
So one of my biggest one was umUnited States versus versus
Roger Geronio and I testifiedback in 2006 in that one and I
investigated it.
It was about a couple of yearsinvestigation in front of it.
I think it was like 2004 to2006-ish.
He committed the crime, if Iremember correctly, somewhere in
(11:16):
the 2002 to 2003 timeframe.
So you know, when you get intothese things it's not like a
quick investigation and it'sover.
You get kind of into them for along time, and so I would work
on these longer investigationsthat would tend to go to trial,
like that one, and in thatparticular case I would.
(11:37):
I got up on the stand and spokeabout you know this was the
logic bomb, which was a piece ofcomputer code that deleted data
, and I demonstrated how it wasspread on these computers and
how it only could have been himand all that kind of stuff, and
so that was probably my largestcase.
The other ones were things likeintellectual property theft, um
(11:57):
, like I talked about earlier,but my last case was probably
the one that you know just if Ican be vulnerable for a moment
probably made me switch, whichwas a murder case, and that was
completely different than allthe other cases that I did,
because I looked at computersfrom the defense and the
prosecution.
I was retained by the defense,but it was just a completely
(12:21):
different experience from the.
I mean there's still electroniccomponents, so it's electronic
crime in a way, because therewere messages sent, but the true
bread and butter electroniccrimes of stealing credit cards
and insider gone wrong.
It just was a completelydifferent ball of wax, and that
was about the time I decided,you know what, maybe I should
get that PhD.
(12:41):
And I looked around and I stillhad trouble finding some
universities in 2016 that taughttrue cybercrime.
Most of them would teachpolicy-related cybercrime stuff,
and I'm a very technical person, I code in most languages and
just doing policy stuff wasn'twhere I wanted to go with my
(13:04):
career.
But Dakota State University hadthis program online that was
for they call it cyberoperations, which is like cyber
crime basically, or computersecurity is how I think of it.
There's cyber operations andthere's cyber defense, and
operations, I guess, is moretechnical, and that's the one
that I did, and I ended up inthat one writing a dissertation
(13:26):
on how to classify malware usingartificial intelligence, and
this is before LLMs.
This is like 2019.
So I had to use things liketensors and the really nerdy and
geeky stuff that we don'treally talk about nowadays.
We just say LLM and everybodyknows what we talk about, but it
(13:47):
was like the very low levelhigh torch tensors that I ran
this data through and basicallywas able to classify them and
then so that's, that's the um,uh, endpoint.
And then in 2019, I switchedover to network.
So I wanted to give you thatlay of the land of we can talk
about investigations, if youlike, we can talk about endpoint
malware dissection, detection,or we can talk about what all
(14:11):
this stuff looks like on thenetwork, which is what I do now,
which is developing algorithmsat scale for think like
universities, really largeuniversities, writing algorithms
that'll say, hey, this trafficout of all your university
traffic, this thing here is abeacon and you need to look at
it.
And that's what I do now.
Speaker 1 (14:29):
Yeah, it's really fascinating.
So you, when you were gettingyour PhD, I was like just
getting into getting my master'sin cybersecurity and I saw that
exact same thing where it wasall policy, it was all theory,
right, there was no hands-onkeyboard.
Um, and the the one uhuniversity that I that I stuck
(14:51):
with, I I went with, uh, capitaltechnology university sounds
like it's one of those you knowschools that are going to be
like you know you're going to begetting a refund from, because
it was a scam, you know, orwhatever it was right, but but
it was extremely hands-on, like,literally, you know we'd have
four, four, five hour classes orsomething like that.
(15:12):
You know four hour, five hour.
And the first half you knowyou're talking about.
You know the logic behind it,right, the thought process
behind.
Okay, like, I'm going to dorecon on this network, this is
what I'm looking for.
You know they'll show you maybea little bit of the like, the
history of it, right, like whereit started and now the tools
that you're going to use andwhatnot.
And then the second half, likeyou're actually doing it.
(15:33):
You know, like they, they walkyou through setting up your home
lab, right, what does that looklike what do I even need?
You know, and you're actuallylaunching all these different
attacks.
You know the forensics one Ithought was actually really
interesting and I actually didpretty well in it.
I wish I just had a betterprofessor, which they now have a
(15:54):
world-class professor in it.
So I'm like even just temptedto go and take that class alone,
you know, just to get the info.
But it was really fascinatingfor me to see how to see like
kind of both sides right, likehow you can go back and you can
basically pull anything from acomputer's history you know in
(16:16):
the registry, whatever it mightbe, and you can follow it all
the way along.
And then you know immediatelywhere my mind goes is well, how
do I get around that?
How do I start manipulatingthat sort of thing?
I guess that's that degeneratehacker mindset in me.
That's like, well, it workslike that.
But what do I do over here?
(16:37):
It's a really fascinating world.
And now I'm getting my PhD fromthat same university world.
And now I'm getting my PhD fromthat same university and it is
uh, it's a struggle trying to,trying to do it while having a
little kid, you know, at homeit's like, um, yeah, I should
have done it when I was younger,honestly, yeah.
Speaker 2 (16:57):
I thought that too and I also thought there was a
definitely a difference where Icould appreciate more stuff
because I had experience,especially in the real world
experience.
If I went away master's PhD job, it would have, I think, been a
completely different experiencethan having you know over a
decade of, you know, real worldexperience in, because it's just
(17:26):
, you had a more well-roundedand well-rounded view of things
and you had examples of thingsthat you've done in the past and
that you could work, you hit,you have you have your interests
kind of fleshed out by then.
You know there's a lot ofthings that I was very grateful
that I did when I was older.
But I agree with you, my kidswere teenagers when I did it.
It was just like trying to dropthem off at things.
(17:47):
You know their events, and thencome back and then get this
paper done and it was a juggleand I, once you get it done,
you'll be so happy and you'll go.
I'll never do that again, yeah.
Speaker 1 (17:55):
Yeah it's.
It's like it's taken me forever.
I mean, it took me like a wholesemester just to figure out
what I wanted to do it on, youknow.
And then, like I don't know,I've run into so many roadblocks
and you know, the biggest issue, the biggest issue by far with
achieving it which I canunderstand why most don't even
complete it right Is all ofschool you're taught okay, the
(18:19):
test is on this, right, thepaper is on this, we want five
pages on this.
This is how I want the fivepages to look.
Like right, and you kind ofjust fit in the idea into that
mold.
In the PhD, it's like no, go,choose what you want to do it on
, get approval to do it.
Hopefully there's enough infothere for you to do a PhD on it,
(18:44):
right, you have to figure outhow you're going to test it,
like what the methods are,everything You're starting from
scratch.
I mean your PhD, you know,could be 20, 40 pages, which I'm
already on like page 40, right,and I'm like not even getting
started on my research or it canbe hundreds of pages long.
That's a crazy variation.
(19:07):
Um, that you know, all ofschool basically doesn't prepare
you for, and maybe it's becausethey don't expect anyone to
like go and get a PhD, exceptfor, you know, 1% of the
population that goes throughschooling.
Maybe, I don't know what thatis.
Speaker 2 (19:23):
There was a really good quote that one of my
professors or advisors said tome one time, which was your
bachelor years we tell you whatthe question is and then we tell
you what the answer is.
In your master's, we tell youwhat the question is and then
you tell us what the answer is.
And then when you get to thephd, you tell us what the
question is and you tell us whatthe answer is, and I thought
(19:46):
that fits perfectly with myexperience and even now with the
research team that I'm on.
There are things that I willjust say I'm going to write a
detection for this and nobodywill ask you, nobody will tell
you how to do it or anything,and it mirrors that process that
you learn in your PhD, which isokay, Go out there, do your
literature review and do this.
(20:07):
All this other stuff and a lotof it is self-motivated that
when you get on research teamslike this with people with PhDs
and so forth at least in myexperience, no one there's very.
There's probably just as manythings that I'll just pick up
and do just out of curiosity asthings where the company says,
hey, we'd like to develop thisthing, so it's kind of nice and
(20:30):
it does very much.
At least my experience is verymuch mirrored what I experienced
in the PhD of having somefreedom to explore and come up
with different things.
Speaker 1 (20:40):
That's interesting.
No-transcript, or that.
(21:11):
You're, you know, investigatingthe case to some extent.
What is that like?
Like, how do you get, how doyou get verified to do that?
Speaker 2 (21:20):
well, let's see.
So it's not like I jumped intothe job.
It kind of just was part of thejob where I would do
investigations.
And then let's say, you do 100best investigations and some of
them might involve court.
What a vast majority of whatyou investigate.
If you find something,somebody's going to settle
(21:41):
either criminal or civil and youdon't really see court.
It's just kind of to developsome data so that they can start
making arguments and thensomebody will settle right, but
then there's that smallpercentage that the sides won't
agree.
You know, either be a criminal,where they're like, hey, I'm
going to trial, or it's civiland they're like we're not going
to pay that much, and theydecide to go to trial.
And in those cases I've alreadybeen on the case when it went
(22:05):
to court.
So I was just the.
I was, I was just the person.
There was nobody else that didit.
So it was me and the processitself is pretty interesting and
I haven't done it since 2015.
So if there's any lawyerslistening or anything like that,
you know it may have changed,but this is, this has been my
experience.
So you'll typically get a caseand you know I'll work for a
(22:26):
company and that company willsay hey, we're getting.
You know, we landed a case.
It's um, let me think of onethat we did.
Well, let me let me take thedronio case.
So it was probably 2003 ishwhen the company I worked for
retained that case and I gotassigned to it and it basically
started with okay, we think thisperson deleted all the data on
(22:51):
the UBS Payne Webber's tradingsystem, so people couldn't trade
stocks because back then, Imean it was computers were a lot
harder to come by than they arenow and it was like each office
had this server and the serverwas responsible for processing
these stock trades and stufflike that.
So if you go in and knock oneof those out, in theory they
can't trade.
Now, in reality, what happenedwas they just took out paper and
(23:13):
they continued to trade.
It was slower and they didn'tget as much done, but you know,
that's how they did it in thepast and that's what they did
when these systems went down.
And so in that particular case,they said, hey, these systems
were deleted.
You, there's no hard drives oranything you can look at, but we
(23:34):
got these tape backups of whatthey looked like right before
they went bad and I was like,okay, so you have to go down the
whole path of I have thesetapes and I got to figure out
what format they are, I got tofigure out what tape drive I
need to get into, how to restorethem, what programs they use,
and there's like, you see,there's like all these steps you
gotta do just to get to thedata, and it's like you have the
data at that point and youstart doing an investigation and
(23:55):
that particular one, I was ableto pinpoint it down to a
particular user which ended upbeing the person that was
arrested anyways.
So what happened was once youcome up with your conclusions.
So in the legal world there'stwo types of witnesses.
There's a fact witness, whichyou could say things like I saw
the red car go through the light.
(24:16):
And then there's an expertwitness who can give you
opinions, which would be it wasmy opinion, the person was drunk
when they went through thelight.
You know what I'm saying whenit's like you have to apply some
knowledge and critical thinkingto it, whereas fact you can
just that's it.
You can only say the facts.
So, as an expert witness,there's this whole process that
(24:36):
happens with this, with theseopinions, and you start it by
writing a report, a physicallike paper report, and you give
it to the attorneys and thenthey'll give it to the other
side and then they'll get anexpert and then that expert will
go through there and tell youwhy you're wrong and how could
you ever think this way.
You're completely wrong.
And so you'll have youtypically have some opposing
(24:57):
expert that will be sayingeverything that you did is wrong
, and in almost every case I canthink of that I've been in,
there's been that opposing youknow expert on the other side.
So once the report is done,typically and this happens more
in civil but you'll be pulledinto a deposition, which is you
have to give live testimony andit's not in front of a jury like
(25:20):
a court case is.
But it's just as importantbecause this is where they get
you into your opinions on recordmore on record because it's
coming out of your voice ratherthan your.
I mean, you got your report, butnow you're up there saying it
right, and so you've got thatrecord coming out and that's
going to basically be availablefor them during the trial too.
(25:40):
So even though people arethinking, oh, I'm going to
testify in trial, that's usuallythe very last step in these
things.
So there's things likedepositions, where it's just as
important for the trial, butit's a completely different
environment where there's nojudge, there's just a court
(26:01):
reporter, the lawyer you're withand the lawyer asking you the
questions.
And I've had those be wild.
I've had lawyers throw theirpens in the air because I
wouldn't answer a question theirway you know the way they want
to meet you and all that kind ofstuff.
And it's just wild becausethere's just no there.
There's no judge and jury, sothey can pretty much do
everything.
But what's on the record is onthe record.
So when the guy throws a pen,you have the attorney going.
I don't appreciate you throwingyour pen and stuff like that
(26:23):
just to get it, just to get iton record.
So if you're at the depositionpoint and you've given your
deposition and you they stillhaven't settled, whatever it is
the criminal or civil case thentypically you'll go to trial.
And then you know, by the pointyou know you're going to trial,
you've had a lot of thiswriting the report, doing the
(26:43):
depositions and all that kind ofstuff to get there.
So it's not really a surpriseor you're not really coming up
with your ideas at the lastmoment or anything.
You know it's months and monthsand months prior that you've
written this report and it'skind of like going to trial,
sort of putting the bow on thepackage where you're, you're now
just basically saying what's inyour report and then it's left
(27:05):
up to a jury or a judge to, youknow, find if you're what you're
saying is believable or not.
Speaker 1 (27:13):
That's interesting.
It's more of, I guess, likeright place, right time.
You're at the right companythat gets that sort of thing and
you're in the right role to beable to handle that sort of
situation, which is fascinating,right, Because I would assume
you would learn a whole lot,probably a whole lot more than
(27:33):
you would expect.
You know because you're nowlooking through, you know
different ways of doingdifferent things and the you
know the background tasks of youknow the bits and the bytes,
right, like what's going on youknow over here and how does all
that tie together?
Like what enabled them to youknow, and how does all that tie
together?
Like what enabled them to, youknow, delete that entire, an
(27:55):
entire database, which waspretty crazy that that happened.
And now they're back to tradingwith paper.
I don't even know how thatworks, but I could imagine I
worked for a financial servicesyou know firm like that before
that did a lot of trading and,man, if something happened to
that data, I mean they mightfire your entire floor just
(28:17):
because you were on it.
Speaker 2 (28:19):
Well, this might give you a chuckle then, because
when you're an expert witness, alot of times the lawyers will
only give you just the piece ofthe case that you need to know,
so that way you won't be askedquestions about stuff that you
just have no business knowingabout, right?
So I you know, it's kind oflike they kept horse blinders on
me where I could just look atthese.
All I knew about up until trialwere these tapes, right.
(28:43):
So like that was my view ofthis whole story.
But because I was an expertwitness, I actually got to sit
and listen to the whole case andall the other witnesses that
went up there, and so what Ifound was there was a logic bomb
set off by this particularadministrator and I found it in
his home directory.
I found it on his homecomputers and stuff.
And I thought, wow, you knowthat really.
(29:03):
That really tied everythingtogether, right.
And I'm sitting there and thenI'm listening to the secret
service agent and he goes ohyeah, we, we found a printed
copy of that logic bomb on hisnightstand next to where he
slept.
It's like you almost didn'teven need me at that point
because you had it in his housenext to where he slept, which
was just.
I've never heard of that, I'venever seen anything like that.
Speaker 1 (29:24):
Since it's just crazy that's crazy that someone
printed out right.
I mean, what's the purpose ofthat?
You're gonna hang it up on yourwall, so, like I did that I
have no idea, but it was.
Speaker 2 (29:36):
That's wild.
I'm looking at this.
I'm like I just built myopinion on all this electronic
evidence and then there's, likethis, one physical piece of
paper that I had no idea was outthere that basically said this
guy definitely knew about thislogic, bob, wow that's pretty
crazy.
Speaker 1 (29:51):
It's, um, yeah, that's really fascinating.
So you so you went from thereinto more research.
Did the PhD kick off before yougot into security research, or
was it the same time, sort ofthing?
Speaker 2 (30:08):
Pretty close to the same time.
I did that murder case in 2015,but I was my whole life I've
been doing software developmentstuff.
So when I was an investigatorand I'd work with other
investigators, I would writetools.
For instance, to do pcicompliance.
You have to search for creditcard numbers and at the time I
was a pci investigator, I didn't.
There were no tools so Ideveloped an extension for our
(30:32):
forensics tools to search acrossit.
So my interest has always beento write these things that help
make one person look like 10, orat least help five people do
their jobs easier than if theywere to do it without it, and so
that research part was alwaysthere for me.
And at the end of that 2015 era, when I started my PhD in 2016,
(30:53):
that's when I switched it was apretty hard switch from doing
just general investigations tomalware analysis.
So, specifically, I startedfocusing on malware.
At that point, even though Idealt with them in
investigations, it was more likeI was like a general medical
doctor in a way, and what I wasdoing is saying I now want to
(31:13):
look at cancer research, thatthat was the equivalent of the
computer switch of what I made.
Speaker 1 (31:19):
Hmm, Okay, that makes sense.
So then you know, talk to meabout being a security
researcher.
What is that like?
Is your company kind of justlike setting you wild on on a
topic, right and and figuringout what's going on over there,
or what is it like?
Speaker 2 (31:36):
Sometimes Sometimes I do, and sometimes I come up
with them on my own.
Sometimes they give me an ideaas they run with it.
Sometimes they already have anidea of what they want.
So you'll have these likecompany ideas where they'll say
hey, like for instance withCoralite, we have this part that
I helped write calledapplication identification,
which looks at connections onyour network and tries to tell
(31:59):
you something about them.
So, for instance, if you weregoing to GitHub to get some
source code, this algorithm thatI wrote would notify you and
said oh, that connection there,that's GitHub and oh, this
connection over here, this isgoing to LastPass and this
connection over here is going tomicrosoft office and generic
things like that.
The company will tend to cometo me and say, hey, we want
something that will do this.
(32:20):
And that was one example wherethey basically said I want this
idea and I just sort of ran withit and figured out the ways to
do the detections and alert theuser and stuff like that.
Some other ones like if youfollow some of my postings, I
will quite often post adetection for a malware family
(32:40):
those I just pick up.
So a lot of times I'll becrunching big, big data sets
with, you know think, biguniversities, all their
connection logs, you knowlooking for patterns and stuff
like that.
So I have little gaps of timewhere I'm waiting for computers
to do stuff.
So one of those gaps of time Iwent to anyrun and started
looking through their they havelike this top 100 or top
(33:05):
whatever number it is list it'sa giant list of these are all
the submissions that we have andthey basically rank them from
this is the most submittedsample to the least submitted
sample.
So when I get bored, instead ofdoing a crossword puzzle or
something like that, I go tothat list and say, all right, I
don't see a detection for thismalware family, and then I'll
(33:25):
start poking into their pcaps,which they allow freely that you
can download them out of thatnetwork section, and I'll start
looking at, looking for c2 orwhatever it is that that mauer
family will do, and then I'llwrite a detection on it in zeek
and then I'll publish it to theopen source community.
But we also will put it in theproduct.
So it really does vary from youhave a vision of something to
(33:48):
hey, that's a great thing thatyou developed.
We didn't even think of doingthat.
So when you're a securityresearcher, I find there's a lot
more um freedom to do thosethings, especially if you have a
track record of doing them.
Before.
When I did the first, firstcouple malware ones, I didn't
really know how to go aboutdoing it.
(34:08):
I didn't know the best way ofdoing it.
But now that I've been doing itfor about a year and I've done
I don't know, know, maybe like10 or so malware families, you
know, it's just like it's just apart of what we do.
Now Nobody really thinks aboutit.
I just will go, pick up anotherpiece of malware and then I'll
write a blog about it, send itto our marketing and then I'll
go back to my day job.
Speaker 1 (34:27):
Hmm, you know what.
What does?
What does malware-baseddetections in AI mean?
I feel like we're getting intoa place with AI where we're kind
of opening the diving into thisAI thing.
(34:55):
None of them know what's goingon, right?
They have no security controlsaround it, they have no concept
of security around it, and youknow it starts going into a
place where you know, well, howdo you know that they're not
mining our data to go and sellit to someone else or make their
model better, right, to go andsell it to someone else or make
their model better, right?
(35:16):
Or how do you not know thatyou're not feeding you know a
piece of malware that'sunderneath their code, that
they're not even thinking ofthemselves, that they, you know,
somehow got infected with thispiece of malware and now it's,
you know, maybe siphoning dataoff of over to, like you know,
the Chinese model.
What is it like?
Seek, seek, deep, deep, seek,deep, seek, deep, seek, right?
What does that look like, evenusing AI to do malware
(35:37):
detections?
Because at some point it'sgoing to figure out the
detections that you're writing,maybe the logic behind it, and
it'll extrapolate on that.
So what's your thoughts on that?
Speaker 2 (35:50):
Well, I started playing around with LLMs several
months ago and what I've beenlearning, developing with LLMs
to do certain parts of my jobfeels like it's so foundational.
It was probably equal to when Ilearned Python.
It was like it's just a newgiant tool I can do stuff with.
And, like you said, when youuse it as a tool, you really
(36:15):
need to know everything that'sgoing on.
And I'm I'm saying this with asmile on my face which is, if
you go to chat gpt, right, andyou just start putting personal
data in there, I'm sure theyhave some kind of privacy policy
and they have some kind ofprocesses behind there, but you,
as a user, do you really know?
You really know, right?
I mean, you got to kind of trustthem, right and then on the
(36:36):
other hand, um, deep seek, onthe other hand, has a chat
application like chat gpt, and II haven't used it, I've just
read about it so far.
I use chat gpt a lot to docomparisons, but from what I
read they, yeah, we're sendingyour data over to Chinese
servers and I would assume thatit's probably going to be used
(36:58):
as a data training set.
Because, from what I understand, deepseq was actually built
because there were exportrestrictions on AI technology.
So there was a group that says,well, we can't use this really
powerful technology to basicallycrunch these things out like
the big Facebooks and Googlescan, so we have to find a more
(37:25):
cost efficient way of doing thisbut still getting great results
.
Supposedly, they have a waythat's a lot cheaper than the
other people like the meta, themetas out there and so forth,
and you know that's.
You got to assume that whenyou're sending that data and
they're trying to train thesemodels to be bigger, better and
faster, to be, you know,basically, the it's like the
moon race, right, it's likethere's I would.
(37:47):
I would think that they wouldtake your data and use it in the
training model.
And now, why I think this isinteresting was because there
was a blog article that we wrotejust a couple days before I
started practice or just playingwith chat GPT to see what it
could do and stuff.
And so I started asking itquestions about just research
(38:09):
that I had been doing.
And I asked it a question aboutdetecting something that we'd
done recently.
But I didn't I didn't sayanything about who I was or
anything and it went and foundthe article that we wrote on
that particular thing and ittold me it's like you want to
use this fine command.
I wrote that fine command.
And it was telling me it was avery, very unique fine command
(38:29):
that when I saw it I was like,yeah, I definitely wrote that
fine command Cause I use, likecertain directory tokens and
stuff in there.
And so, yeah, you got prettymuch have to assume that
anything you send to a serviceis probably going to be used as
training at some point.
And you know there's a lot, aton of celebrities that are
upset about that because youknow their likeness and all that
(38:49):
kind of stuff.
So my solution to the problemis this if you want to use an
llm, there's a lot of opensource software out there.
There's one in particularcalled olama, o-l-l-l-a-m-a, you
put it on your computer I knowit runs on Mac and Linux, I
think it runs on Windows and youcan pull down most of these
(39:10):
models that are used in thesechat applications and you can
run it all local on yourcomputer and not send data
anywhere.
So, for instance, the DeepSeekpeople released DeepSeek a week
or two before they releasedtheir actual application that
you have on your phone, and soif you want to play with
DeepSeek and you're worriedabout sending your data over
there and you want to see whatit does, you can run it locally
(39:31):
on your computer with Ollama andrun the DeepSeek R1 and ask it
questions and at least not worryabout that data going anywhere.
Speaker 1 (39:39):
Huh, wow, that's really interesting.
That's fascinating.
It's, yeah, we're in a weirdplace where it's difficult to
check how your data is beingused in those models, right,
especially with the Chinesemodels and whatnot, right, like
(40:01):
I read some article it was along time ago that surpasses any
other.
You know countries, you knowlike nationality or or you know
(40:22):
urge to do better for thecountry itself, like they really
instill it in their people,right, and so they talked about,
like, even the risk of likehiring chinese nationals that
have spent, you know, decades inamerica.
They talk about the risksbecause it's like, yeah, they've
been here, they probably loveamerica, but you know, the love
(40:46):
that they have for china willgreatly go over, above and
beyond what they have foramerica.
And so you have to like reallyvet their loyalties and whatnot
and almost expect them to go theChina route if they needed to,
to make sure that China overcamewhatever difficulty it might go
(41:06):
through, right, and that'sreally fascinating because I
don't even I like barely view,like being an American citizen.
I mean, I'm very much all forAmerica, right, but I don't know
, I feel like that's still adifferent level, that's still
something completely different.
And so now we're kind of goinginto a place where these models
(41:27):
will be used against us and wetrained them.
You know that's a.
That's a.
I feel like that's somethingthat no one really wants to talk
about right now.
Speaker 2 (41:35):
Yeah, we're.
I think we're still figuringout what these things can do,
because almost daily I willthink of an idea and I'm like I
wonder what the llm will say tothis and I'll put it in and I'll
be completely surprised to belike I can't believe it.
It was so spot on.
I mean, there's plenty of timesit's not, but there are other
times where I'm like there's noway it's going to get this and I
put a question in and it comesback and I'm like, oh my gosh, I
(41:57):
just didn't expect that.
And I would say, to add on whatyou're saying, I, about 15 or so
years ago, I did an expertwitness case where it was theft
of intellectual property.
It was criminal.
If I remember right, I think itwas criminal, it was department
of justice, so I think had tobe criminal.
And what I learned is there's awhole like the cultures are
(42:19):
different.
Even so, business wise, inAmerica we write contracts and
if you and I have a contract,you know in a perfect world we
would abide by that contract andif one of us broke it we would
sue the other person and theywould get you know the situation
and get rectified side, whereasin a particular case that I was
(42:42):
an expert witness for.
I found that, or they explainedto me that in China you'll have
contracts, but it's more like asuggestion than it is what it
is in the U?
S.
So in that particular case itwas kind of like you said, where
there was this American companythat built some software and
they wanted to go reap thereward of being able to sell it
in other countries like Chinaand it's more difficult, I think
(43:05):
, at least back then, to sell inChina.
You had to have an office inthat country and all sorts of
stuff.
Well, what they did is theytook that software, put it on a
server over there and put it inChina, and then they were like,
oh, you're in America, you can'tdo anything, this software is
ours now, yeah, and basicallyyou know, owned it, and what
happened was the person that wasstill in the U?
S.
That was the person that wenton.
(43:26):
You know that I was an expertwitness in his case, he was the
defendant.
So you know that that threat hasbeen there for probably a
couple of decades, where it'slike we want to be able to sell
our product, either a physicalor software product, and there's
this great market over there,but if we sell it over there,
we're running the risk of beingreverse engineered or stolen or
(43:49):
something like that.
So, yeah, it's.
I think that that older styleof intellectual property theft
that we saw for so many years isjust now kind of translating
into llms, where llms will justeat up as much data as it can in
order to train itself, andthat's kind of, in a way, like
taking other people'sintellectual property if you, if
(44:11):
it's not already publicinformation right, yeah, I uh,
you know, I, I work for a largeautomotive manufacturer and we
have a whole Chinese divisionand everything else like that.
Speaker 1 (44:25):
And we were talking about I think it was my
architect was talking about youknow, different security
controls in China and stuff likethat that we had.
And I just looked at my CISOand I said we're in China.
And he said, yep.
I was like so none of this evenmatters.
Then they have everything thatwe could ever have over there.
(44:45):
He goes yep, and I said, arethey at least different cars?
He said no.
I was like, okay, so they haveeverything that we sell.
Then and the discussion endedright there.
It's very safe to assume we havea presence in china.
You know, china has a policywhere if it's in china, it's the
(45:05):
, the republic of china's uh,property, basically right, and
there's nothing that you can do.
And so now it's it's turninginto like ai models and
consuming all the data that itcomes that comes with.
And you know, we're gettinginto a place where it's almost
like these models are going tostart writing themselves, if
(45:25):
they're not already, and wedon't know about it.
Right?
I mean, is that kind of how yousee it too, because we're going
into a place where, you know Imean even Chad GPT was saying
that they're running out of datato train their model.
I mean, they're running out ofdata in the knowable world.
How insane is that.
Speaker 2 (45:42):
It is.
It's crazy the part that I satback and when I tell people,
especially that are out, not inthe computer world, and I say,
listen, so I run these chatGPT-like models, pick like
DeepSeek or whatever I actuallylike 5.4.
That's Microsoft's model.
It's very good withcybersecurity questions, so I've
been using that one and I'llrun that locally.
(46:04):
But I'll run these prompts andI'll get some data out and I'll
be like that's not exactly whatI wanted.
So what I'll do is I'll take myprompt that I usually send to
my, my five, my local model, andI'll go to chat GPT and I'll
(46:25):
say, listen, this thing isproducing some data that looks
like this.
But I really want it to looklike this.
How can I make this better?
And so I have one chat actuallymaking my prompts better for my
other chat and it actuallyworks.
Speaker 1 (46:30):
It's crazy, wow.
Yeah, for you know, for a lotof the PhD research that I'm
doing, I'm doing a literaturereview right now, so I'm using
Brock and Chad GPT prettyheavily, you know, finding the
right papers, finding the rightquotes even within the paper,
tying it all together.
(46:50):
It's really fascinating to seethe differences, because there
will typically be, you know,maybe 80%, like if I say, say,
find me the top 10 articles on,you know, quantum requirements
for quantum encryption to work,maybe seven or eight of the
articles will be the exact same.
You know same sources andeverything.
But then it's those outliers.
(47:12):
You know that it's that itcategorizes something higher
above another or adds indifferent articles and whatnot.
Um, and it's it's interesting tosee it just pulled together
because it's like the new.
It's like the new version ofthe search engine.
I don't go to google.
I didn't go to google like onetime to find any of these
articles.
(47:32):
I literally just went to thislom and I mean it probably knows
more my, it probably knowsexactly what I'm researching and
, you know, has its owndissertation in the background
ready to go.
Speaker 2 (47:44):
Ask it.
Ask ChatGPT what it's learnedabout you.
That's an interesting response.
So I'll I'll ask it things.
I'll just think of randomthings.
I'll be like what's thefunniest thing that I've told
you?
What's the coolest networkingthing that I've shown you?
What's the coolest coding thingthat I've shown you?
What's the coolest coding thingthat I've worked on?
And it'll weirdly summarizestuff that you did like 30 days
(48:07):
ago.
Speaker 1 (48:07):
Oh wow, I'll have to.
I'll have to do that Like assoon as we're off here.
It's pretty cool, that's reallyfascinating.
It's interesting too.
And then there's like a wholepoisoning aspect of these models
, you know, like where I mean Idon't want to say that Google's
model was poisoned, Right, butthe way that it was coded was
(48:30):
obviously very biased towardsone like subsect of the
population in a very odd way,right.
Like I mean, someone gave itthe prompt like give me a
picture of a Nazi, and it had,like an African-American person
you know as a Nazi, and a Naziuniform was like that.
That would never happen, right.
But I'm thinking of it in termsof when you ask it for some
(48:53):
type of critical information andit gives you the wrong answer
intentionally, it makes it seemlike that's the right answer.
It gives you different sourcesfor it.
Maybe it even created thosesources, you know, and so now
we're having limited ways ofeven verifying or validating
that information.
You know, like the Google modelwas very obviously incorrect.
It needed some tweaking.
(49:14):
Google admitted it right.
Admitted it right.
But what if you don't know?
Like you legitimately don'tknow, like for my phd research,
some of this stuff Ilegitimately don't know.
It could tell me the wrongthing and I wouldn't know it and
I would include it in myresearch, not knowing that it's
wrong and I mean you have yourphd right, so your resources are
(49:36):
.
Could be potentially hundreds ofcitations, right, Hundreds of
articles that you pull this infofrom.
Who's going to go and check 200articles that you claim to be
in your research?
It's very small, right.
And now we're turning into thisthing that's being informed by
(49:56):
a model that's potentially beinginfluenced.
You know in some other way thatwe're not seeing.
Speaker 2 (50:02):
Yeah.
It it whatever input they give,it is whatever biases it's
going to have.
So it's it's.
I've looked at a lot of thosemodels through a llama and you
know I could.
It's almost like when you seethese models it's kind of like
meeting a new person because youask it a question and you ask a
technical question and one willbe like really good at you know
(50:22):
, bulleting it out and stuff.
You ask another one and it'll befree form when it tells you and
it's really strange but it'slike I just uh, I just had a
brain fart there for a second.
Sorry, these, uh, oh.
So the the models themselves, Ifind each one will have its own
strength and weaknesses.
So the LAMA model not to beconfused with O-LAMA, but the
(50:45):
LAMA model written by Meta isvery, very good at just natural
language narrative responses.
Where I found the Microsoftmodel its bias in a way, is it's
really good at, in my opinion,at cybersecurity questions.
I can ask it pretty up to datecybersecurity questions and it
it gets it, whereas with youknow the meta one, I have to
(51:08):
tell it a little bit aboutcybersecurity for it to get it.
Model that came out the deepseek r1.
When you run it it lookscompletely different.
It looks like it reasons.
It doesn't just say here's myanswer.
It basically says okay, inorder to answer this question,
I've got to do this, and then itdoes that and then it's like,
(51:29):
okay, now that I know this, Igotta know that.
And then it does it and it'slike it tells you how it reasons
and then it goes.
Here's my answer.
Yeah, it's really kind of crazy.
If you get a chance to play,then it goes.
Here's my answer.
Yeah, it's really kind of crazy.
If you get a chance to playwith it, it's.
It's a lot of fun to see it.
It it looks like a humanreasoning when it answers you.
But my point being is it's youknow there are, there are bad
(51:51):
biases out there, but there arealso these biases in a way of
which model can answer thequestion.
You're going to ask itcorrectly and I just know if I'm
going to ask it a cybersecurityquestion, I'm going to pull out
Microsoft's model.
If I'm going to ask somethingnarrative related, where I need
a nice passage of English text,I'll probably go to metas and
(52:11):
ask it the question.
So you just kind of you got toplay with them and kind of
understand them.
It's interesting.
Speaker 1 (52:18):
I mean it's almost like there's some sort of bias
towards, maybe, who was creatingit.
Right, like you think about howMeta's model, you know, does
better with text, or you knowthe flow of language and whatnot
.
Well, I mean that's probablybecause they probably trained it
on.
You know Facebook, right, andthey see how everyone talks and
(52:40):
the differences betweendifferent countries and the
language and the slang termsused and everything else.
Like that.
I mean there's nothing else onthe planet that would even come
close to that.
Right, like, especially ifyou're using that model and
you're saying, oh okay, two,three billion people use this
thing every single day.
Go learn everything that youcan from it.
(53:02):
How are they talking andwhatnot.
And then you know, like I kindof brought up before the
differences between Grok andChatGPT.
Right, like, there are smallerdifferences.
It seems like almost ChatGPT ismore I don't want to say like
legalese, but more professionalin how it answers things, and
Grok is a little bit more, youknow, free form.
(53:25):
Almost to some extent It'llgive you the same info, but it
responds on a little bit moreconversational than than chat
GPT does in some ways.
Right, but it's, it'sfascinating.
We're moving into an area where, you know, I've been saying it
for a couple of years now righton the podcast, where one of the
(53:45):
top emerging fields is AIsecurity.
And then the next question ispeople ask me, well, what's AI
security?
And I literally my onlyresponse is I don't know.
Like we're still trying tofigure it out.
Speaker 2 (53:58):
I was reading an interesting thing that people
are trying to fight back againstthese ai web crawlers that,
will you know, pull in, pull intons of web data and use it for
training, and there is said thatthey're basically using a tar
pit like tar pit, like the, thedefensive tactic that was used
in spam years ago.
(54:18):
They're basically doing the samething where, where these AI
bots will come to your websiteand start crawling for data and,
instead of you saying, allright, I'm going to give you my
normal data when you recognizeit, you go, I'm going to give
you a bunch of links and you'rejust going to keep traversing
down these links and I'm justgoing to make you busy forever
because you're bothering mywebsite.
(54:39):
And going to make you busyforever because you're bothering
my website.
Wow, and yeah, it was prettyinteresting.
I was just like I was sittingthere and I was like I never
even thought, like I would nevereven have thought to do that.
Right now, you, you have peoplethat are basically building
defenses against these ai botcrawlers because it's hitting
websites millions of times a dayto pull in new, new data wow, I
mean it's gonna make peopleit's right there.
(55:01):
It's like a job right.
Speaker 1 (55:02):
I mean that, yeah, that just made a job of how
someone has to defend againstthis stuff yeah, I, maybe I need
to go like form an ai securitycompany where all we do is ai
security stuff.
I'm sure it would be likebought up.
You know, by x or someone, likeimmediately as soon as you do
it, you just got to have onecustomer.
That'll be interesting.
But you know, by X or someonelike immediately as soon as you
do it, you just got to have onecustomer.
That'll be interesting.
(55:23):
But you know, keith, this, uh,this conversation has been
really fascinating.
I really enjoyed it and I needto have you back on.
I mean, I think it would bereally interesting to have you
back on, maybe even in a couplemonths, right, like with the,
with the rapid pace that AI is,uh, is you know, evolving, it
might be beneficial to have youback on soon.
Speaker 2 (55:43):
Absolutely, I'd be happy to, and whenever you need
anything, just let me know.
Speaker 1 (55:46):
Yeah, absolutely.
Well, you know, before I letyou go, how about you tell my
audience you know where they canfind you if they want to reach
out to you, where they can findyour company.
And I saw that you have apodcast.
I don't know if you're stilldoing it, but all that info I've
been doing it off and on.
Speaker 2 (56:00):
Basically, I just make fun of electronic crime
criminals and how they getcaught.
But if you want to get a holdof me probably the best way you
can hit my blog first.
That's just dr, as in doctorkeithjonescom, and if you don't
spell Keith a lot, it's E-I, notI-E, and that would be my
personal blog.
My work, the company I work for, is named Correlate and they're
(56:21):
Correlatecom.
They have network sensors thatdetect a lot of these things
that I was able to talk to youabout today, and you can find me
on LinkedIn.
I'm always on LinkedIn.
That's probably the number onesocial media place that I'll
visit the most, and just do asearch on there for Keith Jones
and I.
I'm in Maryland.
Speaker 1 (56:40):
I'll I should pop up there pretty close to the top.
Awesome Sounds, great.
Well, thanks everyone.
I hope you enjoyed this episode.
Thank you much.
how's it going?
Keith, it's great to get you onthe podcast.
You know, I think we've beenplanning this thing for a while
and you know I got burnt outtowards the end of last year.
It just seems to happen everysingle year I get burnt out.
And then I went and, you know,decided to get sick with the
rest of my family for all of theholidays.
So I'm glad that we can finally, you know, get you on here and
(00:22):
have a great conversation.
Speaker 2 (00:28):
Thanks for having me.
I'm glad you're feeling bettertoo, yeah.
Speaker 1 (00:29):
It's a.
It was a blur last year too, Iagree with that, yeah, yeah, I
mean last year, I mean literally, you know it was December and I
mean my head was still in May,it was still like thinking about
the summer and stuff.
You know it's I don't know.
I think when you have kids,like everyone says that it like
speeds things up, you know, butlike it's so true, because every
single day is, you know, I havea two year old, right?
(00:50):
So every single day is herlearning a new word or
interacting with us a differentway, or, you know, giving us
attitude a new way, right, Likeit's all, it's all fun and new.
And when you have somethinglike that, I mean it just, it
just flies by.
Speaker 2 (01:03):
And when you have something like that.
I mean it just.
It just flies by Absolutely.
Mine are older, mine are likehigh school and college and my
one recommendation is going tobe don't blink, cause it's like
mine came home for um uh, youknow, the holiday break and he
just left and I think we're thenext time he comes home is going
to be close to Thanksgiving.
So it's like with a little man,just just adore that time.
Speaker 1 (01:25):
Yeah, you know.
So it's like, yeah, with alittle man, just just adore that
time.
Yeah, you know it's.
It's fascinating, right, how,how priorities change and
whatnot.
And I was always the kind of guythat when a when a better
opportunity would come along,like I would just take it.
I wouldn't even think about itor anything like that, right.
Well, now I've been remote forfive years, right, and I have a
two-year-old and I have a secondone on the way, due end of
(01:47):
April timeframe.
And fairly recently I hadsomeone offer me a job, better
opportunity, right, really bigfinancial institution worldwide,
a lot more money, right.
And I mean, five years ago Iwould have taken that without,
without thinking about it.
Right, the big problem was thatI'd have to go into the office
(02:10):
three days a week and I satthere and I thought about, like
the pay difference isn't worthme not being home when my kid
gets here, right, like it's thatit's a weird, it's a weird
predicament, because even 10, 15years ago, right, when I was
getting into the field, I meanit wasn't even, wasn't even an
option, right, like it was, likeyou're going into the office,
(02:31):
that's what it is, you know, ifyou can't live with it like get
out, get out of the industry,you know.
But now, like we have thatprivilege of being like, nah,
I'd rather, I'd rather be remote, to save the two, three hours
of commute, you know, just so Icould have those hours with my
kid.
Speaker 2 (02:49):
Yeah, the company I work for right now.
Their name is Coralite, andmost of us, so I work in
Coralite Labs, which is aresearch branch of Coralite.
We make the algorithms thatyou'll see on the sensors and we
have researchers that areliterally scattered all over the
world and it's really kind ofneat because me, being on the
(03:10):
east coast, I'll wake up at acertain time and I'll see a
certain set of people on butthey kind of fade off and then
midday a new set of people comeon and it's like we almost have
24 hour coverage, because evenfor a small team, because we're
scattered so much, and it makesit really beneficial for a lot
of things that we do, like keepthe processes running and doing
research around the clock andstuff like that.
Speaker 1 (03:27):
Yeah, it's way more beneficial and it makes so much
more sense for technology,people, people in IT, to be able
to be remote, right?
I mean, if the pandemic taughtus anything, it is that there's
literally no reason for us to bein the office, like literally
zero.
Right, you can't use the excusethat technology isn't there,
(03:49):
it's not available.
You know we don't have it.
You know the VPN only takes somany, so many users at a time,
right, like, we don't even havethat big of a trunk, right,
whatever those excuses were, allof that is completely gone.
You know them saying, oh, thebusiness won't survive.
It's like, well, you survivedfor two and a half, three years
during the pandemic witheveryone a hundred percent
(04:11):
remote.
What it like.
What are you doing now?
Right, like you know, Iregularly have an Amazon
recruiter reaching out to meevery single, every year, every
six months, something like thatRight Seeing if I want to come
on.
And this past year, every sixmonths, something like that
right Seeing if I want to comeon.
And this past year, or maybe youknow, four, four ish months ago
, one reached out to me and saidyou know, it was like two weeks
(04:33):
after the five day work week.
You know, push from Amazon,right when they announced it.
And I mean, he reached out tome and the very first thing I
said was, no, I don't think I'minterested because I wouldn't go
into the office five days aweek.
And first thing I said was no,I don't think I'm interested
because I wouldn't go into theoffice five days a week.
And that's really sayingsomething, because even a year
ago, I interviewed with a teamthere and they were three days
in the office and I asked him Iwas like well, where's everyone
(04:53):
located?
And the hiring manager was likeoh yeah, some people are in New
Jersey, new York, washington,seattle.
You'll be the only one inChicago.
And I said so I have to go in towork remote.
He said oh, you don't have tothink about it like that.
I was like no, that's what itis.
My entire team is around theglobe right In different time
(05:17):
zones.
I'm never going to see themface to face in person.
You just told me I won't, right?
So what's the point?
And he couldn't answer thatquestion.
Obviously, I ended up not goingthere.
But you know, it's like thesecompanies, they're really
pushing hard for people to goback in when, like you just said
, it's actually moreadvantageous for you to be
(05:39):
remote, because now you have24-7 coverage on a team that
otherwise would never have 24-7coverage.
And if a service provider evenclaimed that they could provide,
you know, coverage on threatresearch or security researching
, they're lying to you becausethat doesn't really exist.
Speaker 2 (05:56):
Yeah, one of the in our area, around the Washington
DC area, there's an interestinglittle twist because I, like you
, I see these things come across.
You know LinkedIn and stuff,and around here it tends to be
government contracting type ofwork and most of that type of
work tends to be classified.
So it's like if you seesomething that's I don't know
(06:17):
from like Raytheon or some youknow really big defense
contractor, you almost knowimmediately you got to go in for
that and do the classified work.
And you know, for some people,if they're not, they're not
living in DC, they're not goingto be able to do that.
Speaker 1 (06:30):
Yeah, yeah, no, that's that's a really good
point.
I mean, that's the.
I feel like that's probably oneof the very few industries as a
whole.
That it's you know.
It's forever in the office.
There's nothing you could doabout it.
And it's you know it's foreverin the office.
There's nothing you can doabout it, and it's you know
within within reason too right.
You wouldn't want someone takingtheir work home, like that's
illegal, yeah.
(06:51):
Then then it was started like awhole other conversation of
well, if it's legal now, thenwhy was Edward Snowden illegal
and you know all that, all thatsort of stuff, right.
But you know, keith, I reallywant to dive into your
background right.
How you got started, what madeyou want to get started right,
(07:23):
because you know I view securityas kind of like a crazy, not
necessarily wild west, but ofdive into security.
So how did you find your wayinto IT and how did that evolve
into security?
Speaker 2 (07:45):
for it but also kind of lucky.
So if we go back way back, Imean in the eighties, I can
remember programming on VIC-20sand Commodores and stuff like
that as a kid and I was justalways really, really interested
in computers and computers.
You know, if you're not as oldas me and you're watching,
listening to this, they were notall that common back then.
So if you had a VIC-20 aroundthat was a big deal as a kid.
So you know I got to learningprogramming and stuff like that
and then by the time I switchedto college there wasn't a
(08:08):
computer security anything inthe 90s, the early 90s, when I
went to college and when I didthat I knew I wanted to be in
computers and the easiest thingor the most matching major for
that was computer engineeringand electrical engineering.
So I did a double major forbachelor's in that and then by
the time I was done with thebachelor's I still kind of
(08:28):
looked around and I was like Idon't really want to design
chips the rest of my life likeCPUs and stuff like that.
I don't want to write operatingsystem code.
I know I'm interested in thisother stuff that during that
time I was working at the schoolof criminal justice too.
So I had exposure to cybercrime and just all sorts of
(08:49):
crime.
I even worked with them onetime help them clean up a blood
splatter room, so it was kind ofit was a really neat as a
computer guy.
It was a really neat job tohave back in the nineties.
So I got to see the lawenforcement side of side of
things which kind of went withcomputer crime.
But still nobody was calling itcomputer crime back then.
(09:11):
And so I graduated with mymaster's in 99 and I figured you
know what, I don't want to geta PhD yet because there's
nothing in here that's computercrime related.
If I did it it would have beenstraight electrical engineering
or computer science back in 99and it just didn't fit.
So I went to work and, um, Istarted out because I had a
(09:32):
programming background.
I helped work on some securitysoftware, some like log
correlation type of tools Ithink Splunk, but like way
before Splunk, just to you know,match web logs to a login and
not logins, but like yourfirewall logs and stuff like
that.
And in that I ended uptransitioning into doing
(09:52):
investigations.
So when I say investigations Imean very loosely, not just, you
know, remote attacker breakinginto your network but also your
internal employee gone rogue, orsometimes it was civil stuff
where one company would accuseanother company of stealing
their source code and I'd haveto look at it and say yes or no.
This is, you know, this is orisn't the same as what you think
(10:14):
it is, and I worked in thatspace for quite a while,
somewhere between like two earlytwo thousands through about 20,
16 ish.
So I did investigations forquite a while and in there I I
had experience doing experttestimony in federal court, um
(10:35):
criminal and civil.
I've done state courts liketennessee and some other states
that off the top of my head I'mnot remembering right now, but
you know a handful of otherstates and um it gave me
experience with a lot ofdifferent types of cases.
So some of the cases were,let's see, for the testimony.
A lot of the testimony tendedto be insider cases.
(10:56):
So one of my biggest one was umUnited States versus versus
Roger Geronio and I testifiedback in 2006 in that one and I
investigated it.
It was about a couple of yearsinvestigation in front of it.
I think it was like 2004 to2006-ish.
He committed the crime, if Iremember correctly, somewhere in
(11:16):
the 2002 to 2003 timeframe.
So you know, when you get intothese things it's not like a
quick investigation and it'sover.
You get kind of into them for along time, and so I would work
on these longer investigationsthat would tend to go to trial,
like that one, and in thatparticular case I would.
(11:37):
I got up on the stand and spokeabout you know this was the
logic bomb, which was a piece ofcomputer code that deleted data
, and I demonstrated how it wasspread on these computers and
how it only could have been himand all that kind of stuff, and
so that was probably my largestcase.
The other ones were things likeintellectual property theft, um
(11:57):
, like I talked about earlier,but my last case was probably
the one that you know just if Ican be vulnerable for a moment
probably made me switch, whichwas a murder case, and that was
completely different than allthe other cases that I did,
because I looked at computersfrom the defense and the
prosecution.
I was retained by the defense,but it was just a completely
(12:21):
different experience from the.
I mean there's still electroniccomponents, so it's electronic
crime in a way, because therewere messages sent, but the true
bread and butter electroniccrimes of stealing credit cards
and insider gone wrong.
It just was a completelydifferent ball of wax, and that
was about the time I decided,you know what, maybe I should
get that PhD.
(12:41):
And I looked around and I stillhad trouble finding some
universities in 2016 that taughttrue cybercrime.
Most of them would teachpolicy-related cybercrime stuff,
and I'm a very technical person, I code in most languages and
just doing policy stuff wasn'twhere I wanted to go with my
(13:04):
career.
But Dakota State University hadthis program online that was
for they call it cyberoperations, which is like cyber
crime basically, or computersecurity is how I think of it.
There's cyber operations andthere's cyber defense, and
operations, I guess, is moretechnical, and that's the one
that I did, and I ended up inthat one writing a dissertation
(13:26):
on how to classify malware usingartificial intelligence, and
this is before LLMs.
This is like 2019.
So I had to use things liketensors and the really nerdy and
geeky stuff that we don'treally talk about nowadays.
We just say LLM and everybodyknows what we talk about, but it
(13:47):
was like the very low levelhigh torch tensors that I ran
this data through and basicallywas able to classify them and
then so that's, that's the um,uh, endpoint.
And then in 2019, I switchedover to network.
So I wanted to give you thatlay of the land of we can talk
about investigations, if youlike, we can talk about endpoint
malware dissection, detection,or we can talk about what all
(14:11):
this stuff looks like on thenetwork, which is what I do now,
which is developing algorithmsat scale for think like
universities, really largeuniversities, writing algorithms
that'll say, hey, this trafficout of all your university
traffic, this thing here is abeacon and you need to look at
it.
And that's what I do now.
Speaker 1 (14:29):
Yeah, it's really fascinating.
So you, when you were gettingyour PhD, I was like just
getting into getting my master'sin cybersecurity and I saw that
exact same thing where it wasall policy, it was all theory,
right, there was no hands-onkeyboard.
Um, and the the one uhuniversity that I that I stuck
(14:51):
with, I I went with, uh, capitaltechnology university sounds
like it's one of those you knowschools that are going to be
like you know you're going to begetting a refund from, because
it was a scam, you know, orwhatever it was right, but but
it was extremely hands-on, like,literally, you know we'd have
four, four, five hour classes orsomething like that.
(15:12):
You know four hour, five hour.
And the first half you knowyou're talking about.
You know the logic behind it,right, the thought process
behind.
Okay, like, I'm going to dorecon on this network, this is
what I'm looking for.
You know they'll show you maybea little bit of the like, the
history of it, right, like whereit started and now the tools
that you're going to use andwhatnot.
And then the second half, likeyou're actually doing it.
(15:33):
You know, like they, they walkyou through setting up your home
lab, right, what does that looklike what do I even need?
You know, and you're actuallylaunching all these different
attacks.
You know the forensics one Ithought was actually really
interesting and I actually didpretty well in it.
I wish I just had a betterprofessor, which they now have a
(15:54):
world-class professor in it.
So I'm like even just temptedto go and take that class alone,
you know, just to get the info.
But it was really fascinatingfor me to see how to see like
kind of both sides right, likehow you can go back and you can
basically pull anything from acomputer's history you know in
(16:16):
the registry, whatever it mightbe, and you can follow it all
the way along.
And then you know immediatelywhere my mind goes is well, how
do I get around that?
How do I start manipulatingthat sort of thing?
I guess that's that degeneratehacker mindset in me.
That's like, well, it workslike that.
But what do I do over here?
(16:37):
It's a really fascinating world.
And now I'm getting my PhD fromthat same university world.
And now I'm getting my PhD fromthat same university and it is
uh, it's a struggle trying to,trying to do it while having a
little kid, you know, at homeit's like, um, yeah, I should
have done it when I was younger,honestly, yeah.
Speaker 2 (16:57):
I thought that too and I also thought there was a
definitely a difference where Icould appreciate more stuff
because I had experience,especially in the real world
experience.
If I went away master's PhD job, it would have, I think, been a
completely different experiencethan having you know over a
decade of, you know, real worldexperience in, because it's just
(17:26):
, you had a more well-roundedand well-rounded view of things
and you had examples of thingsthat you've done in the past and
that you could work, you hit,you have you have your interests
kind of fleshed out by then.
You know there's a lot ofthings that I was very grateful
that I did when I was older.
But I agree with you, my kidswere teenagers when I did it.
It was just like trying to dropthem off at things.
(17:47):
You know their events, and thencome back and then get this
paper done and it was a juggleand I, once you get it done,
you'll be so happy and you'll go.
I'll never do that again, yeah.
Speaker 1 (17:55):
Yeah it's.
It's like it's taken me forever.
I mean, it took me like a wholesemester just to figure out
what I wanted to do it on, youknow.
And then, like I don't know,I've run into so many roadblocks
and you know, the biggest issue, the biggest issue by far with
achieving it which I canunderstand why most don't even
complete it right Is all ofschool you're taught okay, the
(18:19):
test is on this, right, thepaper is on this, we want five
pages on this.
This is how I want the fivepages to look.
Like right, and you kind ofjust fit in the idea into that
mold.
In the PhD, it's like no, go,choose what you want to do it on
, get approval to do it.
Hopefully there's enough infothere for you to do a PhD on it,
(18:44):
right, you have to figure outhow you're going to test it,
like what the methods are,everything You're starting from
scratch.
I mean your PhD, you know,could be 20, 40 pages, which I'm
already on like page 40, right,and I'm like not even getting
started on my research or it canbe hundreds of pages long.
That's a crazy variation.
(19:07):
Um, that you know, all ofschool basically doesn't prepare
you for, and maybe it's becausethey don't expect anyone to
like go and get a PhD, exceptfor, you know, 1% of the
population that goes throughschooling.
Maybe, I don't know what thatis.
Speaker 2 (19:23):
There was a really good quote that one of my
professors or advisors said tome one time, which was your
bachelor years we tell you whatthe question is and then we tell
you what the answer is.
In your master's, we tell youwhat the question is and then
you tell us what the answer is.
And then when you get to thephd, you tell us what the
question is and you tell us whatthe answer is, and I thought
(19:46):
that fits perfectly with myexperience and even now with the
research team that I'm on.
There are things that I willjust say I'm going to write a
detection for this and nobodywill ask you, nobody will tell
you how to do it or anything,and it mirrors that process that
you learn in your PhD, which isokay, Go out there, do your
literature review and do this.
(20:07):
All this other stuff and a lotof it is self-motivated that
when you get on research teamslike this with people with PhDs
and so forth at least in myexperience, no one there's very.
There's probably just as manythings that I'll just pick up
and do just out of curiosity asthings where the company says,
hey, we'd like to develop thisthing, so it's kind of nice and
(20:30):
it does very much.
At least my experience is verymuch mirrored what I experienced
in the PhD of having somefreedom to explore and come up
with different things.
Speaker 1 (20:40):
That's interesting.
No-transcript, or that.
(21:11):
You're, you know, investigatingthe case to some extent.
What is that like?
Like, how do you get, how doyou get verified to do that?
Speaker 2 (21:20):
well, let's see.
So it's not like I jumped intothe job.
It kind of just was part of thejob where I would do
investigations.
And then let's say, you do 100best investigations and some of
them might involve court.
What a vast majority of whatyou investigate.
If you find something,somebody's going to settle
(21:41):
either criminal or civil and youdon't really see court.
It's just kind of to developsome data so that they can start
making arguments and thensomebody will settle right, but
then there's that smallpercentage that the sides won't
agree.
You know, either be a criminal,where they're like, hey, I'm
going to trial, or it's civiland they're like we're not going
to pay that much, and theydecide to go to trial.
And in those cases I've alreadybeen on the case when it went
(22:05):
to court.
So I was just the.
I was, I was just the person.
There was nobody else that didit.
So it was me and the processitself is pretty interesting and
I haven't done it since 2015.
So if there's any lawyerslistening or anything like that,
you know it may have changed,but this is, this has been my
experience.
So you'll typically get a caseand you know I'll work for a
(22:26):
company and that company willsay hey, we're getting.
You know, we landed a case.
It's um, let me think of onethat we did.
Well, let me let me take thedronio case.
So it was probably 2003 ishwhen the company I worked for
retained that case and I gotassigned to it and it basically
started with okay, we think thisperson deleted all the data on
(22:51):
the UBS Payne Webber's tradingsystem, so people couldn't trade
stocks because back then, Imean it was computers were a lot
harder to come by than they arenow and it was like each office
had this server and the serverwas responsible for processing
these stock trades and stufflike that.
So if you go in and knock oneof those out, in theory they
can't trade.
Now, in reality, what happenedwas they just took out paper and
(23:13):
they continued to trade.
It was slower and they didn'tget as much done, but you know,
that's how they did it in thepast and that's what they did
when these systems went down.
And so in that particular case,they said, hey, these systems
were deleted.
You, there's no hard drives oranything you can look at, but we
(23:34):
got these tape backups of whatthey looked like right before
they went bad and I was like,okay, so you have to go down the
whole path of I have thesetapes and I got to figure out
what format they are, I got tofigure out what tape drive I
need to get into, how to restorethem, what programs they use,
and there's like, you see,there's like all these steps you
gotta do just to get to thedata, and it's like you have the
data at that point and youstart doing an investigation and
(23:55):
that particular one, I was ableto pinpoint it down to a
particular user which ended upbeing the person that was
arrested anyways.
So what happened was once youcome up with your conclusions.
So in the legal world there'stwo types of witnesses.
There's a fact witness, whichyou could say things like I saw
the red car go through the light.
(24:16):
And then there's an expertwitness who can give you
opinions, which would be it wasmy opinion, the person was drunk
when they went through thelight.
You know what I'm saying whenit's like you have to apply some
knowledge and critical thinkingto it, whereas fact you can
just that's it.
You can only say the facts.
So, as an expert witness,there's this whole process that
(24:36):
happens with this, with theseopinions, and you start it by
writing a report, a physicallike paper report, and you give
it to the attorneys and thenthey'll give it to the other
side and then they'll get anexpert and then that expert will
go through there and tell youwhy you're wrong and how could
you ever think this way.
You're completely wrong.
And so you'll have youtypically have some opposing
(24:57):
expert that will be sayingeverything that you did is wrong
, and in almost every case I canthink of that I've been in,
there's been that opposing youknow expert on the other side.
So once the report is done,typically and this happens more
in civil but you'll be pulledinto a deposition, which is you
have to give live testimony andit's not in front of a jury like
(25:20):
a court case is.
But it's just as importantbecause this is where they get
you into your opinions on recordmore on record because it's
coming out of your voice ratherthan your.
I mean, you got your report, butnow you're up there saying it
right, and so you've got thatrecord coming out and that's
going to basically be availablefor them during the trial too.
(25:40):
So even though people arethinking, oh, I'm going to
testify in trial, that's usuallythe very last step in these
things.
So there's things likedepositions, where it's just as
important for the trial, butit's a completely different
environment where there's nojudge, there's just a court
(26:01):
reporter, the lawyer you're withand the lawyer asking you the
questions.
And I've had those be wild.
I've had lawyers throw theirpens in the air because I
wouldn't answer a question theirway you know the way they want
to meet you and all that kind ofstuff.
And it's just wild becausethere's just no there.
There's no judge and jury, sothey can pretty much do
everything.
But what's on the record is onthe record.
So when the guy throws a pen,you have the attorney going.
I don't appreciate you throwingyour pen and stuff like that
(26:23):
just to get it, just to get iton record.
So if you're at the depositionpoint and you've given your
deposition and you they stillhaven't settled, whatever it is
the criminal or civil case thentypically you'll go to trial.
And then you know, by the pointyou know you're going to trial,
you've had a lot of thiswriting the report, doing the
(26:43):
depositions and all that kind ofstuff to get there.
So it's not really a surpriseor you're not really coming up
with your ideas at the lastmoment or anything.
You know it's months and monthsand months prior that you've
written this report and it'skind of like going to trial,
sort of putting the bow on thepackage where you're, you're now
just basically saying what's inyour report and then it's left
(27:05):
up to a jury or a judge to, youknow, find if you're what you're
saying is believable or not.
Speaker 1 (27:13):
That's interesting.
It's more of, I guess, likeright place, right time.
You're at the right companythat gets that sort of thing and
you're in the right role to beable to handle that sort of
situation, which is fascinating,right, Because I would assume
you would learn a whole lot,probably a whole lot more than
(27:33):
you would expect.
You know because you're nowlooking through, you know
different ways of doingdifferent things and the you
know the background tasks of youknow the bits and the bytes,
right, like what's going on youknow over here and how does all
that tie together?
Like what enabled them to youknow, and how does all that tie
together?
Like what enabled them to, youknow, delete that entire, an
(27:55):
entire database, which waspretty crazy that that happened.
And now they're back to tradingwith paper.
I don't even know how thatworks, but I could imagine I
worked for a financial servicesyou know firm like that before
that did a lot of trading and,man, if something happened to
that data, I mean they mightfire your entire floor just
(28:17):
because you were on it.
Speaker 2 (28:19):
Well, this might give you a chuckle then, because
when you're an expert witness, alot of times the lawyers will
only give you just the piece ofthe case that you need to know,
so that way you won't be askedquestions about stuff that you
just have no business knowingabout, right?
So I you know, it's kind oflike they kept horse blinders on
me where I could just look atthese.
All I knew about up until trialwere these tapes, right.
(28:43):
So like that was my view ofthis whole story.
But because I was an expertwitness, I actually got to sit
and listen to the whole case andall the other witnesses that
went up there, and so what Ifound was there was a logic bomb
set off by this particularadministrator and I found it in
his home directory.
I found it on his homecomputers and stuff.
And I thought, wow, you knowthat really.
(29:03):
That really tied everythingtogether, right.
And I'm sitting there and thenI'm listening to the secret
service agent and he goes ohyeah, we, we found a printed
copy of that logic bomb on hisnightstand next to where he
slept.
It's like you almost didn'teven need me at that point
because you had it in his housenext to where he slept, which
was just.
I've never heard of that, I'venever seen anything like that.
Speaker 1 (29:24):
Since it's just crazy that's crazy that someone
printed out right.
I mean, what's the purpose ofthat?
You're gonna hang it up on yourwall, so, like I did that I
have no idea, but it was.
Speaker 2 (29:36):
That's wild.
I'm looking at this.
I'm like I just built myopinion on all this electronic
evidence and then there's, likethis, one physical piece of
paper that I had no idea was outthere that basically said this
guy definitely knew about thislogic, bob, wow that's pretty
crazy.
Speaker 1 (29:51):
It's, um, yeah, that's really fascinating.
So you so you went from thereinto more research.
Did the PhD kick off before yougot into security research, or
was it the same time, sort ofthing?
Speaker 2 (30:08):
Pretty close to the same time.
I did that murder case in 2015,but I was my whole life I've
been doing software developmentstuff.
So when I was an investigatorand I'd work with other
investigators, I would writetools.
For instance, to do pcicompliance.
You have to search for creditcard numbers and at the time I
was a pci investigator, I didn't.
There were no tools so Ideveloped an extension for our
(30:32):
forensics tools to search acrossit.
So my interest has always beento write these things that help
make one person look like 10, orat least help five people do
their jobs easier than if theywere to do it without it, and so
that research part was alwaysthere for me.
And at the end of that 2015 era, when I started my PhD in 2016,
(30:53):
that's when I switched it was apretty hard switch from doing
just general investigations tomalware analysis.
So, specifically, I startedfocusing on malware.
At that point, even though Idealt with them in
investigations, it was more likeI was like a general medical
doctor in a way, and what I wasdoing is saying I now want to
(31:13):
look at cancer research, thatthat was the equivalent of the
computer switch of what I made.
Speaker 1 (31:19):
Hmm, Okay, that makes sense.
So then you know, talk to meabout being a security
researcher.
What is that like?
Is your company kind of justlike setting you wild on on a
topic, right and and figuringout what's going on over there,
or what is it like?
Speaker 2 (31:36):
Sometimes Sometimes I do, and sometimes I come up
with them on my own.
Sometimes they give me an ideaas they run with it.
Sometimes they already have anidea of what they want.
So you'll have these likecompany ideas where they'll say
hey, like for instance withCoralite, we have this part that
I helped write calledapplication identification,
which looks at connections onyour network and tries to tell
(31:59):
you something about them.
So, for instance, if you weregoing to GitHub to get some
source code, this algorithm thatI wrote would notify you and
said oh, that connection there,that's GitHub and oh, this
connection over here, this isgoing to LastPass and this
connection over here is going tomicrosoft office and generic
things like that.
The company will tend to cometo me and say, hey, we want
something that will do this.
(32:20):
And that was one example wherethey basically said I want this
idea and I just sort of ran withit and figured out the ways to
do the detections and alert theuser and stuff like that.
Some other ones like if youfollow some of my postings, I
will quite often post adetection for a malware family
(32:40):
those I just pick up.
So a lot of times I'll becrunching big, big data sets
with, you know think, biguniversities, all their
connection logs, you knowlooking for patterns and stuff
like that.
So I have little gaps of timewhere I'm waiting for computers
to do stuff.
So one of those gaps of time Iwent to anyrun and started
looking through their they havelike this top 100 or top
(33:05):
whatever number it is list it'sa giant list of these are all
the submissions that we have andthey basically rank them from
this is the most submittedsample to the least submitted
sample.
So when I get bored, instead ofdoing a crossword puzzle or
something like that, I go tothat list and say, all right, I
don't see a detection for thismalware family, and then I'll
(33:25):
start poking into their pcaps,which they allow freely that you
can download them out of thatnetwork section, and I'll start
looking at, looking for c2 orwhatever it is that that mauer
family will do, and then I'llwrite a detection on it in zeek
and then I'll publish it to theopen source community.
But we also will put it in theproduct.
So it really does vary from youhave a vision of something to
(33:48):
hey, that's a great thing thatyou developed.
We didn't even think of doingthat.
So when you're a securityresearcher, I find there's a lot
more um freedom to do thosethings, especially if you have a
track record of doing them.
Before.
When I did the first, firstcouple malware ones, I didn't
really know how to go aboutdoing it.
(34:08):
I didn't know the best way ofdoing it.
But now that I've been doing itfor about a year and I've done
I don't know, know, maybe like10 or so malware families, you
know, it's just like it's just apart of what we do.
Now Nobody really thinks aboutit.
I just will go, pick up anotherpiece of malware and then I'll
write a blog about it, send itto our marketing and then I'll
go back to my day job.
Speaker 1 (34:27):
Hmm, you know what.
What does?
What does malware-baseddetections in AI mean?
I feel like we're getting intoa place with AI where we're kind
of opening the diving into thisAI thing.
(34:55):
None of them know what's goingon, right?
They have no security controlsaround it, they have no concept
of security around it, and youknow it starts going into a
place where you know, well, howdo you know that they're not
mining our data to go and sellit to someone else or make their
model better, right, to go andsell it to someone else or make
their model better, right?
(35:16):
Or how do you not know thatyou're not feeding you know a
piece of malware that'sunderneath their code, that
they're not even thinking ofthemselves, that they, you know,
somehow got infected with thispiece of malware and now it's,
you know, maybe siphoning dataoff of over to, like you know,
the Chinese model.
What is it like?
Seek, seek, deep, deep, seek,deep, seek, deep, seek, right?
What does that look like, evenusing AI to do malware
(35:37):
detections?
Because at some point it'sgoing to figure out the
detections that you're writing,maybe the logic behind it, and
it'll extrapolate on that.
So what's your thoughts on that?
Speaker 2 (35:50):
Well, I started playing around with LLMs several
months ago and what I've beenlearning, developing with LLMs
to do certain parts of my jobfeels like it's so foundational.
It was probably equal to when Ilearned Python.
It was like it's just a newgiant tool I can do stuff with.
And, like you said, when youuse it as a tool, you really
(36:15):
need to know everything that'sgoing on.
And I'm I'm saying this with asmile on my face which is, if
you go to chat gpt, right, andyou just start putting personal
data in there, I'm sure theyhave some kind of privacy policy
and they have some kind ofprocesses behind there, but you,
as a user, do you really know?
You really know, right?
I mean, you got to kind of trustthem, right and then on the
(36:36):
other hand, um, deep seek, onthe other hand, has a chat
application like chat gpt, and II haven't used it, I've just
read about it so far.
I use chat gpt a lot to docomparisons, but from what I
read they, yeah, we're sendingyour data over to Chinese
servers and I would assume thatit's probably going to be used
(36:58):
as a data training set.
Because, from what I understand, deepseq was actually built
because there were exportrestrictions on AI technology.
So there was a group that says,well, we can't use this really
powerful technology to basicallycrunch these things out like
the big Facebooks and Googlescan, so we have to find a more
(37:25):
cost efficient way of doing thisbut still getting great results
.
Supposedly, they have a waythat's a lot cheaper than the
other people like the meta, themetas out there and so forth,
and you know that's.
You got to assume that whenyou're sending that data and
they're trying to train thesemodels to be bigger, better and
faster, to be, you know,basically, the it's like the
moon race, right, it's likethere's I would.
(37:47):
I would think that they wouldtake your data and use it in the
training model.
And now, why I think this isinteresting was because there
was a blog article that we wrotejust a couple days before I
started practice or just playingwith chat GPT to see what it
could do and stuff.
And so I started asking itquestions about just research
(38:09):
that I had been doing.
And I asked it a question aboutdetecting something that we'd
done recently.
But I didn't I didn't sayanything about who I was or
anything and it went and foundthe article that we wrote on
that particular thing and ittold me it's like you want to
use this fine command.
I wrote that fine command.
And it was telling me it was avery, very unique fine command
(38:29):
that when I saw it I was like,yeah, I definitely wrote that
fine command Cause I use, likecertain directory tokens and
stuff in there.
And so, yeah, you got prettymuch have to assume that
anything you send to a serviceis probably going to be used as
training at some point.
And you know there's a lot, aton of celebrities that are
upset about that because youknow their likeness and all that
(38:49):
kind of stuff.
So my solution to the problemis this if you want to use an
llm, there's a lot of opensource software out there.
There's one in particularcalled olama, o-l-l-l-a-m-a, you
put it on your computer I knowit runs on Mac and Linux, I
think it runs on Windows and youcan pull down most of these
(39:10):
models that are used in thesechat applications and you can
run it all local on yourcomputer and not send data
anywhere.
So, for instance, the DeepSeekpeople released DeepSeek a week
or two before they releasedtheir actual application that
you have on your phone, and soif you want to play with
DeepSeek and you're worriedabout sending your data over
there and you want to see whatit does, you can run it locally
(39:31):
on your computer with Ollama andrun the DeepSeek R1 and ask it
questions and at least not worryabout that data going anywhere.
Speaker 1 (39:39):
Huh, wow, that's really interesting.
That's fascinating.
It's, yeah, we're in a weirdplace where it's difficult to
check how your data is beingused in those models, right,
especially with the Chinesemodels and whatnot, right, like
(40:01):
I read some article it was along time ago that surpasses any
other.
You know countries, you knowlike nationality or or you know
(40:22):
urge to do better for thecountry itself, like they really
instill it in their people,right, and so they talked about,
like, even the risk of likehiring chinese nationals that
have spent, you know, decades inamerica.
They talk about the risksbecause it's like, yeah, they've
been here, they probably loveamerica, but you know, the love
(40:46):
that they have for china willgreatly go over, above and
beyond what they have foramerica.
And so you have to like reallyvet their loyalties and whatnot
and almost expect them to go theChina route if they needed to,
to make sure that China overcamewhatever difficulty it might go
(41:06):
through, right, and that'sreally fascinating because I
don't even I like barely view,like being an American citizen.
I mean, I'm very much all forAmerica, right, but I don't know
, I feel like that's still adifferent level, that's still
something completely different.
And so now we're kind of goinginto a place where these models
(41:27):
will be used against us and wetrained them.
You know that's a.
That's a.
I feel like that's somethingthat no one really wants to talk
about right now.
Speaker 2 (41:35):
Yeah, we're.
I think we're still figuringout what these things can do,
because almost daily I willthink of an idea and I'm like I
wonder what the llm will say tothis and I'll put it in and I'll
be completely surprised to belike I can't believe it.
It was so spot on.
I mean, there's plenty of timesit's not, but there are other
times where I'm like there's noway it's going to get this and I
put a question in and it comesback and I'm like, oh my gosh, I
(41:57):
just didn't expect that.
And I would say, to add on whatyou're saying, I, about 15 or so
years ago, I did an expertwitness case where it was theft
of intellectual property.
It was criminal.
If I remember right, I think itwas criminal, it was department
of justice, so I think had tobe criminal.
And what I learned is there's awhole like the cultures are
(42:19):
different.
Even so, business wise, inAmerica we write contracts and
if you and I have a contract,you know in a perfect world we
would abide by that contract andif one of us broke it we would
sue the other person and theywould get you know the situation
and get rectified side, whereasin a particular case that I was
(42:42):
an expert witness for.
I found that, or they explainedto me that in China you'll have
contracts, but it's more like asuggestion than it is what it
is in the U?
S.
So in that particular case itwas kind of like you said, where
there was this American companythat built some software and
they wanted to go reap thereward of being able to sell it
in other countries like Chinaand it's more difficult, I think
(43:05):
, at least back then, to sell inChina.
You had to have an office inthat country and all sorts of
stuff.
Well, what they did is theytook that software, put it on a
server over there and put it inChina, and then they were like,
oh, you're in America, you can'tdo anything, this software is
ours now, yeah, and basicallyyou know, owned it, and what
happened was the person that wasstill in the U?
S.
That was the person that wenton.
(43:26):
You know that I was an expertwitness in his case, he was the
defendant.
So you know that that threat hasbeen there for probably a
couple of decades, where it'slike we want to be able to sell
our product, either a physicalor software product, and there's
this great market over there,but if we sell it over there,
we're running the risk of beingreverse engineered or stolen or
(43:49):
something like that.
So, yeah, it's.
I think that that older styleof intellectual property theft
that we saw for so many years isjust now kind of translating
into llms, where llms will justeat up as much data as it can in
order to train itself, andthat's kind of, in a way, like
taking other people'sintellectual property if you, if
(44:11):
it's not already publicinformation right, yeah, I uh,
you know, I, I work for a largeautomotive manufacturer and we
have a whole Chinese divisionand everything else like that.
Speaker 1 (44:25):
And we were talking about I think it was my
architect was talking about youknow, different security
controls in China and stuff likethat that we had.
And I just looked at my CISOand I said we're in China.
And he said, yep.
I was like so none of this evenmatters.
Then they have everything thatwe could ever have over there.
(44:45):
He goes yep, and I said, arethey at least different cars?
He said no.
I was like, okay, so they haveeverything that we sell.
Then and the discussion endedright there.
It's very safe to assume we havea presence in china.
You know, china has a policywhere if it's in china, it's the
(45:05):
, the republic of china's uh,property, basically right, and
there's nothing that you can do.
And so now it's it's turninginto like ai models and
consuming all the data that itcomes that comes with.
And you know, we're gettinginto a place where it's almost
like these models are going tostart writing themselves, if
(45:25):
they're not already, and wedon't know about it.
Right?
I mean, is that kind of how yousee it too, because we're going
into a place where, you know Imean even Chad GPT was saying
that they're running out of datato train their model.
I mean, they're running out ofdata in the knowable world.
How insane is that.
Speaker 2 (45:42):
It is.
It's crazy the part that I satback and when I tell people,
especially that are out, not inthe computer world, and I say,
listen, so I run these chatGPT-like models, pick like
DeepSeek or whatever I actuallylike 5.4.
That's Microsoft's model.
It's very good withcybersecurity questions, so I've
been using that one and I'llrun that locally.
(46:04):
But I'll run these prompts andI'll get some data out and I'll
be like that's not exactly whatI wanted.
So what I'll do is I'll take myprompt that I usually send to
my, my five, my local model, andI'll go to chat GPT and I'll
(46:25):
say, listen, this thing isproducing some data that looks
like this.
But I really want it to looklike this.
How can I make this better?
And so I have one chat actuallymaking my prompts better for my
other chat and it actuallyworks.
Speaker 1 (46:30):
It's crazy, wow.
Yeah, for you know, for a lotof the PhD research that I'm
doing, I'm doing a literaturereview right now, so I'm using
Brock and Chad GPT prettyheavily, you know, finding the
right papers, finding the rightquotes even within the paper,
tying it all together.
(46:50):
It's really fascinating to seethe differences, because there
will typically be, you know,maybe 80%, like if I say, say,
find me the top 10 articles on,you know, quantum requirements
for quantum encryption to work,maybe seven or eight of the
articles will be the exact same.
You know same sources andeverything.
But then it's those outliers.
(47:12):
You know that it's that itcategorizes something higher
above another or adds indifferent articles and whatnot.
Um, and it's it's interesting tosee it just pulled together
because it's like the new.
It's like the new version ofthe search engine.
I don't go to google.
I didn't go to google like onetime to find any of these
articles.
(47:32):
I literally just went to thislom and I mean it probably knows
more my, it probably knowsexactly what I'm researching and
, you know, has its owndissertation in the background
ready to go.
Speaker 2 (47:44):
Ask it.
Ask ChatGPT what it's learnedabout you.
That's an interesting response.
So I'll I'll ask it things.
I'll just think of randomthings.
I'll be like what's thefunniest thing that I've told
you?
What's the coolest networkingthing that I've shown you?
What's the coolest coding thingthat I've shown you?
What's the coolest coding thingthat I've worked on?
And it'll weirdly summarizestuff that you did like 30 days
(48:07):
ago.
Speaker 1 (48:07):
Oh wow, I'll have to.
I'll have to do that Like assoon as we're off here.
It's pretty cool, that's reallyfascinating.
It's interesting too.
And then there's like a wholepoisoning aspect of these models
, you know, like where I mean Idon't want to say that Google's
model was poisoned, Right, butthe way that it was coded was
(48:30):
obviously very biased towardsone like subsect of the
population in a very odd way,right.
Like I mean, someone gave itthe prompt like give me a
picture of a Nazi, and it had,like an African-American person
you know as a Nazi, and a Naziuniform was like that.
That would never happen, right.
But I'm thinking of it in termsof when you ask it for some
(48:53):
type of critical information andit gives you the wrong answer
intentionally, it makes it seemlike that's the right answer.
It gives you different sourcesfor it.
Maybe it even created thosesources, you know, and so now
we're having limited ways ofeven verifying or validating
that information.
You know, like the Google modelwas very obviously incorrect.
It needed some tweaking.
(49:14):
Google admitted it right.
Admitted it right.
But what if you don't know?
Like you legitimately don'tknow, like for my phd research,
some of this stuff Ilegitimately don't know.
It could tell me the wrongthing and I wouldn't know it and
I would include it in myresearch, not knowing that it's
wrong and I mean you have yourphd right, so your resources are
(49:36):
.
Could be potentially hundreds ofcitations, right, Hundreds of
articles that you pull this infofrom.
Who's going to go and check 200articles that you claim to be
in your research?
It's very small, right.
And now we're turning into thisthing that's being informed by
(49:56):
a model that's potentially beinginfluenced.
You know in some other way thatwe're not seeing.
Speaker 2 (50:02):
Yeah.
It it whatever input they give,it is whatever biases it's
going to have.
So it's it's.
I've looked at a lot of thosemodels through a llama and you
know I could.
It's almost like when you seethese models it's kind of like
meeting a new person because youask it a question and you ask a
technical question and one willbe like really good at you know
(50:22):
, bulleting it out and stuff.
You ask another one and it'll befree form when it tells you and
it's really strange but it'slike I just uh, I just had a
brain fart there for a second.
Sorry, these, uh, oh.
So the the models themselves, Ifind each one will have its own
strength and weaknesses.
So the LAMA model not to beconfused with O-LAMA, but the
(50:45):
LAMA model written by Meta isvery, very good at just natural
language narrative responses.
Where I found the Microsoftmodel its bias in a way, is it's
really good at, in my opinion,at cybersecurity questions.
I can ask it pretty up to datecybersecurity questions and it
it gets it, whereas with youknow the meta one, I have to
(51:08):
tell it a little bit aboutcybersecurity for it to get it.
Model that came out the deepseek r1.
When you run it it lookscompletely different.
It looks like it reasons.
It doesn't just say here's myanswer.
It basically says okay, inorder to answer this question,
I've got to do this, and then itdoes that and then it's like,
(51:29):
okay, now that I know this, Igotta know that.
And then it does it and it'slike it tells you how it reasons
and then it goes.
Here's my answer.
Yeah, it's really kind of crazy.
If you get a chance to play,then it goes.
Here's my answer.
Yeah, it's really kind of crazy.
If you get a chance to playwith it, it's.
It's a lot of fun to see it.
It it looks like a humanreasoning when it answers you.
But my point being is it's youknow there are, there are bad
(51:51):
biases out there, but there arealso these biases in a way of
which model can answer thequestion.
You're going to ask itcorrectly and I just know if I'm
going to ask it a cybersecurityquestion, I'm going to pull out
Microsoft's model.
If I'm going to ask somethingnarrative related, where I need
a nice passage of English text,I'll probably go to metas and
(52:11):
ask it the question.
So you just kind of you got toplay with them and kind of
understand them.
It's interesting.
Speaker 1 (52:18):
I mean it's almost like there's some sort of bias
towards, maybe, who was creatingit.
Right, like you think about howMeta's model, you know, does
better with text, or you knowthe flow of language and whatnot
.
Well, I mean that's probablybecause they probably trained it
on.
You know Facebook, right, andthey see how everyone talks and
(52:40):
the differences betweendifferent countries and the
language and the slang termsused and everything else.
Like that.
I mean there's nothing else onthe planet that would even come
close to that.
Right, like, especially ifyou're using that model and
you're saying, oh okay, two,three billion people use this
thing every single day.
Go learn everything that youcan from it.
(53:02):
How are they talking andwhatnot.
And then you know, like I kindof brought up before the
differences between Grok andChatGPT.
Right, like, there are smallerdifferences.
It seems like almost ChatGPT ismore I don't want to say like
legalese, but more professionalin how it answers things, and
Grok is a little bit more, youknow, free form.
(53:25):
Almost to some extent It'llgive you the same info, but it
responds on a little bit moreconversational than than chat
GPT does in some ways.
Right, but it's, it'sfascinating.
We're moving into an area where, you know, I've been saying it
for a couple of years now righton the podcast, where one of the
(53:45):
top emerging fields is AIsecurity.
And then the next question ispeople ask me, well, what's AI
security?
And I literally my onlyresponse is I don't know.
Like we're still trying tofigure it out.
Speaker 2 (53:58):
I was reading an interesting thing that people
are trying to fight back againstthese ai web crawlers that,
will you know, pull in, pull intons of web data and use it for
training, and there is said thatthey're basically using a tar
pit like tar pit, like the, thedefensive tactic that was used
in spam years ago.
(54:18):
They're basically doing the samething where, where these AI
bots will come to your websiteand start crawling for data and,
instead of you saying, allright, I'm going to give you my
normal data when you recognizeit, you go, I'm going to give
you a bunch of links and you'rejust going to keep traversing
down these links and I'm justgoing to make you busy forever
because you're bothering mywebsite.
(54:39):
And going to make you busyforever because you're bothering
my website.
Wow, and yeah, it was prettyinteresting.
I was just like I was sittingthere and I was like I never
even thought, like I would nevereven have thought to do that.
Right now, you, you have peoplethat are basically building
defenses against these ai botcrawlers because it's hitting
websites millions of times a dayto pull in new, new data wow, I
mean it's gonna make peopleit's right there.
(55:01):
It's like a job right.
Speaker 1 (55:02):
I mean that, yeah, that just made a job of how
someone has to defend againstthis stuff yeah, I, maybe I need
to go like form an ai securitycompany where all we do is ai
security stuff.
I'm sure it would be likebought up.
You know, by x or someone, likeimmediately as soon as you do
it, you just got to have onecustomer.
That'll be interesting.
But you know, by X or someonelike immediately as soon as you
do it, you just got to have onecustomer.
That'll be interesting.
(55:23):
But you know, keith, this, uh,this conversation has been
really fascinating.
I really enjoyed it and I needto have you back on.
I mean, I think it would bereally interesting to have you
back on, maybe even in a couplemonths, right, like with the,
with the rapid pace that AI is,uh, is you know, evolving, it
might be beneficial to have youback on soon.
Speaker 2 (55:43):
Absolutely, I'd be happy to, and whenever you need
anything, just let me know.
Speaker 1 (55:46):
Yeah, absolutely.
Well, you know, before I letyou go, how about you tell my
audience you know where they canfind you if they want to reach
out to you, where they can findyour company.
And I saw that you have apodcast.
I don't know if you're stilldoing it, but all that info I've
been doing it off and on.
Speaker 2 (56:00):
Basically, I just make fun of electronic crime
criminals and how they getcaught.
But if you want to get a holdof me probably the best way you
can hit my blog first.
That's just dr, as in doctorkeithjonescom, and if you don't
spell Keith a lot, it's E-I, notI-E, and that would be my
personal blog.
My work, the company I work for, is named Correlate and they're
(56:21):
Correlatecom.
They have network sensors thatdetect a lot of these things
that I was able to talk to youabout today, and you can find me
on LinkedIn.
I'm always on LinkedIn.
That's probably the number onesocial media place that I'll
visit the most, and just do asearch on there for Keith Jones
and I.
I'm in Maryland.
Speaker 1 (56:40):
I'll I should pop up there pretty close to the top.
Awesome Sounds, great.
Well, thanks everyone.
I hope you enjoyed this episode.
Thank you much.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!