May 26, 2025 • 26 mins
Clint Marsden breaks down a critical cybersecurity report from intelligence agencies including the CSA, NSA, and FBI about the growing threat of AI data poisoning. Learn how malicious actors can hijack AI systems for as little as $60, turning machine learning models against their intended purpose by corrupting training data.
Clint explains the technical concept of data poisoning in accessible terms, comparing it to teaching a child the wrong labels for objects. He walks through the six-stage framework where AI systems become vulnerable, from initial design to production deployment, and covers the ten security recommendations intelligence agencies are now promoting to defend against these attacks.
The episode explores real-world examples of AI systems gone wrong, from shopping bots buying drugs on the dark web to coordinated attacks by online communities. You'll discover practical mitigation strategies including cryptographic verification, secure data storage, anomaly detection, and the importance of "human in the loop" safeguards.
Whether you're a cybersecurity professional, AI developer, or simply curious about emerging digital threats, this episode provides essential insights into protecting AI systems from manipulation and understanding why data integrity has become a national security concern.
Key Topics Covered:
- Split view poisoning and expired domain attacks
- Data sanitization and anomaly detection techniques
- Zero trust principles for AI infrastructure
- The role of adversarial machine learning in cybersecurity
- Why defenders must learn AI as quickly as attackers
The PDF from CISA etc al: https://www.ic3.gov/CSA/2025/250522.pdf
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
Hey everyone.
Welcome back to TLP.
I'm your host, Clint Marsden,and today we're gonna talk about
something that's getting a bitmore airtime and dunno if it's a
bit more scary or if it's justbecoming kind of the norm.
But if you picture this, AIsystems are essentially
everywhere now, right?
(00:22):
They're in our phone, they're inour car security systems,
protecting criticalinfrastructure.
They're everywhere.
All of these systems arelearning AI from some kind of
data.
The AI has not come up with thison its own.
(00:43):
It has like all learning hasbeen given this as a, a data
set.
Of course, if some evil doer,some bond villain.
Has gone and messed with thatdata.
If they've modified it, putthings in there that are
inaccurate.
(01:04):
Then basically this controls howthe AI thinks.
I'm sure you remember a fewyears ago when AI or automated
robots online started to getsome airplay.
I think there was a researcherwho created an AI bot to go and
(01:24):
buy things.
Online and very quickly itstarted to just go and buy drugs
on the dark web.
It was kind of a, kind of an anadvanced shopping bot and it
shows how quickly things canturn without the correct, fine,
fine tuning and without thecorrect guardrails.
(01:47):
And there have been otherinstances of early AI systems
where when they went onto theinternet.
The likes of people from fourchan or the likes of people from
Reddit have decided for onereason or another that they want
to try and take this down andjust fill it with junk and to
fill the training model withjunk data.
(02:09):
Maybe they're anti ai, maybeit's just for the lulls.
Who knows?
But that is what happened.
And most recently we've seen areport that's been created in
collaboration with a lot ofintelligence agencies.
I believe it's been spearheadedby CSA as well as the NSA, the
F-B-I-G-C-H-Q, Australia's a SD.
(02:31):
These are all intelligenceagencies, including other
intelligence agencies fromaround the world.
Was also cool to see somerepresentation from the New
Zealand Cybersecurity Center.
They're now reporting, this isbecoming a lot more prevalent,
the hijacking.
Of AI systems also known aspoisoning, is, is becoming the
(02:52):
new norm.
It is not expensive to perform.
We are talking the cost of, say,a laptop.
In some cases we're looking at,so if a cost of a laptop might
be two and a half thousanddollars, then there are also
cases of this being done forunder$60 or under a hundred
(03:13):
dollars.
As little as a little, as littleas 60.
The training model of of AI islike learning as a child, and
it's like teaching a child howto recognize things.
What we are doing while we'relearning to learn, we look at,
sometimes we look at pictures ofthings, and if we're teaching a
child, uh, or if we're learninganother language, you might
(03:33):
receive an image and you showthem a picture of a cat or a
picture of a dog, and you showedthem this and you say, this is a
dog.
And over time throughrepetition.
That cognitive enhancement, theylearn the difference.
What we're talking about withthis AI poisoning is someone
kind of coming in and, andflipping that around.
(03:56):
The label being this is a catwhen it's actually a dog, and
vice versa.
The technical term here is, isactually called data poisoning.
What we're really doing is,well, we're making a rod for our
own back by by doing this, butwe're just lying to, we're lying
to robots.
Because the AI is in thisimpressionable phase, it doesn't
(04:17):
have any idea that it's beenlied to.
It's, it's developing, it'slearning these capabilities.
It is designed to learn what ithas been taught.
This goes into the memory, andthen unfortunately over time
these issues compound because itis retaining this incorrect
(04:38):
information that it has beentaught.
And then it's making decisionsand delivering output based on
that poisoned information.
The report that CSA havereleased has got six stages
where things can go wrong with,with this data poisoning.
And in a, in a brief summary, itstarts off where, when
(05:02):
organizations are planning thearchitecture and the design of
their AI systems.
And then it finishes at when thesystem is running in production
and it's making real decisionsand it's, it's informing real
decisions made by humans in theloop and humans in the loop, if
you don't know, is just a stepwhere you can run an AI
(05:23):
automation and at some point itstops and waits for someone to a
human to physically verify thatwhat it is proposing to do is
okay.
So this might be useful in a,the con.
I mean, I don't have any contextto give you for something in a
nuclear power plant of anything,anything like that.
But what I'm looking at at themoment is content creation using
(05:47):
ai and before it goes out, Iwanna make sure that it matches
the tone and structure and hasinformation that I would publish
if I was writing it.
And if I need to, I will thenget in there and adjust it.
Accordingly and make sure thatit is delivering the message
that I want.
Because the last thing that I'mtrying to produce is what people
(06:09):
are calling AI slop.
I don't want to produce contentthat's AI slop because it goes
against everything that I'mtrying to do.
I'm trying to produce contentthat is helpful for the reader
or is helpful for the listener.
So that's what Human in the Loopdoes that gives you that, that
kind of last chance before itgoes live.
(06:30):
To verify and make any changesand then you can publish it and
to get around these perceivedproblems.
And I say perceived because Idon't have any physical examples
of them right now, to give you,not to diminish the seriousness
of, of what they are.
The document has provided uswith some frameworks that we can
(06:54):
use.
The first thing is we need toknow where our data's coming
from.
So we're going back to some realfundamentals here.
Like if you download software onthe internet, you see there's an
option to download thecryptographic hash as well.
Just to go into a little bit ofa segue, say you're running
Microsoft have retired this now,but say you're running a red
(07:17):
forest, you're running hardeneddomain controllers in a tiered
environment in a T zeroenvironment, which might contain
your domain controllers andother T zero systems.
Like exchange servers.
Remember those before we went tooffice 3, 6, 5.
Before you install any softwarein this environment, once you've
downloaded the installerpackage, or even if you've
(07:39):
already got the installerpackage, you need to make sure
that it passes the hashingalgorithm to ensure that it
hasn't been modified in transitor has been modified repacked,
loaded with some malware, loadedwith something dodgy in the time
that it's been sitting on yoursystems.
We won't get into, but how doyou make sure that the
(08:00):
cryptographic hash that's beingprovided is correct?
'cause that just creates a bitof a loop and I dunno, honestly,
dunno how to solve that one.
So what they're suggesting isuse a cryptographic hash to
verify the data and you canstill hash the contents of
documents.
It's not only reserved forbinary files, they also talk
(08:20):
about using blockchain tracking.
And if it fails, these checksdon't use it.
This isn't so much of an issue.
If the document is not a hugetome, that will take a long time
to view.
But I recently purchased a bookon AI and red teaming, and
specifically red teaming the AIplatform.
(08:41):
The book is a thousand pages.
It's gonna take me a long timeto get through that book.
It's gonna take me months.
That is where the, using acryptographic hash to verify
that the contents has not beenmodified before, using it as
training data is gonna be quitehelpful.
Second, they're talking againabout maintaining the data
(09:02):
integrity during storage andtransport, and this is, this is
where I kind of disagree in howthey've done this.
It's, this is kind of one point.
Yes.
If we're downloading the contentof the information.
Sure.
We need to verify that thecontents is what we expect it to
(09:22):
be, and then if it's beingstored well before, it's then
used in the training data.
Yes, hash it.
Again, similar to forensicpractices where we might be
running.
Every time the data istransferred or moved or used,
you might decide to run a hashto make sure that it is
maintaining that integrity.
So moving on to number three.
(09:44):
Again, same kind of concepthere.
Employing digital signatures toauthenticate trusted data
revisions, using a CA to ensurethat things are not being
modified in transit.
And I'm just gonna move on tonumber four, which is leverage
trusted infrastructure.
(10:06):
Remember those words?
Zero trust.
If you're still trying to getyour zero trust program up and
running, I understand it's a bitof a beast.
The point here is that they'retalking about providing a secure
enclave for storing your data.
Again, I feel like we are stillon the same point.
Uh, I can't believe that we havefour points of this
(10:28):
recommendation that areessentially bundled into the one
concept.
Keeping it in a secure locationand hashing it and then
verifying it, and then makingsure that it can't be modified
in in transit.
This could be some real strongrecommendations.
This could be from areas of thegovernment that we might not
normally hear from, wherethey're operating in such a
(10:52):
secure environment that theseare the measures that they have
to take.
I kind of feel that these aresome super secure areas.
Or some super securerecommendations that they would
follow, maybe in top secretnetworks.
I've never worked in a topsecret network, so I'm only
guessing, but listening orreading the amount of rigor
(11:14):
that's needed for these types ofscenarios, it seems, it seems
pretty plausible.
Then of course, we're talkingabout user access control,
role-based access control, RAC.
Making sure that only the rightpeople are allowed to access
these things.
That comes down to principle ofleast privilege, something that
(11:36):
I spoke about very, very earlyon in the, one of the, I think
the first four podcast episodeswhen I was doing that NIST
series.
Making sure that only the peoplewho are going to be working with
the system, the, I guess thedigital librarians are the ones
that have the access.
You don't wanna be training theAI again, once you do it, once
(11:57):
you get this first hurdle out ofthe way.
So you wanna make sure thatyou're training it with the
right stuff.
Then of course, encryption,that's a bit of a no brainer
these days.
It's, it's so easy to ensurethat things are encrypted and
that covers not just data atrest, but data in transit.
So data at rest, sitting on thedrive, sitting in a database,
(12:17):
data in transit, going across anetwork, going from cloud hosted
platform where your AI might bestored, even if it's private
cloud.
Or even if it's just across yournetwork, these may not be
problems that you need to dealwith right now, but as the
company expands, these arethings that you will need to
(12:39):
ensure that they're covered.
As this system becomes a bitmore of a mission critical
system for you, it's somethingthat you'll need to have anyway.
And the point is to do this nowso that you're not dealing with
technical debt later.
I've seen technical debt so manytimes and over so many years at
all these differentorganizations.
And I think it's easy to try andavoid doing things right
(13:02):
initially from a place of weneed to get this out.
We're behind on the project.
We want to implement it.
It needs to happen.
Now there might be somedownwards pressure from above to
make things happen.
Maybe there are other reasons.
Competitive advantage.
The sooner we can get this out,the sooner we can take more
market share.
(13:23):
Great.
Let's do that, but let's do itsecurely.
Let's follow these basicfundamental principles right
now.
Hopefully it's why you'relistening to this particular
episode because you wanna knowwhat do we need to do now to
protect us for the future?
Again, number seven, talkingabout storing data securely kind
(13:44):
of done to death by now.
Talking about the use ofcryptographic modules and
encryption.
It's quite interesting actually,because this is a TLP clear.
So you've got TLP, clear green,amber, and red for something
that's A TLP Clear.
It seems to have such a deepresonance with things that may
(14:06):
not be in a TLP clearenvironment or maybe in
documents that you mightgenerally not have access to.
So it's pretty cool that thegovernment is sharing these
types of things with us asprivate citizens.
Moving on to number eight.
Leveraging privacy, preservingtechniques, and then doing
things that will make yourprivacy officer very happy and
(14:28):
very proud of you.
Doing things like data maskingthat is removing sensitive data
with and replacing itinauthentic, but realistic
information basically means whenyou look at the data set, it
still looks normal, but insteadof Clint must, and in the
record, it's.
Tom Jones.
Of course, it needs to be namesthat do not exist in your
(14:51):
customer base to make iteffective.
It's not enough to just move therows around and move someone's
name from row 2000 to row 500.
It needs to be completelyunique.
Number nine is deleting datasecurely, working with physical
hard drives, and then it's timeto dispose of them because they
have failed.
(15:12):
This is important even if thedrive appears to have failed.
Or if you are at end of lifethree years down the line, four
years down the line, and youdecide that it's time to upgrade
these systems, obviously thebest recommendation is to
physically destroy them.
Putting them through a shredderis a common favorite method of
doing that, engaging a thirdparty company.
(15:33):
I used to drill holes in theplatters of hard drives before
we threw them out, and there's alot of discussion on this.
And Rob Lee from Sands.
Talks about it and the, I don'tknow if it's a myth, but there's
the knowledge of an electronmicroscope being used to recover
data.
And this was back in the dayswhen hard drives were only 20
(15:55):
gig.
Uh, now I've heard that thedensity of data is so large that
even an electron microscope isnot gonna help anyone trying.
I guess it depends on yourthreat model.
Uh, if you are worried aboutpeople using an electron
microscope to recover data fromthe drives.
Melt them down, shred them, dowhatever you need to do.
But physical destruction isprobably the best.
(16:18):
Talking about data wipingmethods, the DOD three pass, or
the eight pass, or the 16 passtype wipes, from what I have
read in the research, notnecessary, just a single proper.
Once a once it is wiped, it iswiped.
Not a quick format.
(16:39):
Quick format just markseverything available for
deletion.
That's why it's quick.
The data is still there.
It's not physically there.
Doing a zero of the drive flipsall the bits to zeros, and in
the case that it's already azero, I believe that it flips it
to a one and then back to a zeroto ensure that it's fully gone.
And then lastly, number number10 or step 10 here is to conduct
(17:02):
ongoing data risk assessments.
So using frameworks like theNIST SP 800 dash three R two
haven't heard of the three R twobefore must be a a latest
revision.
The idea here is to evaluate thesecurity landscape, identify
risks, and then prioritize theactions that are relevant to
(17:24):
your organization to minimizebreaches or security incidents.
They then move into somedifferent types of risks and
these risks.
The first one that they'retalking about is a curated web
scale data set.
They make reference to a coupleof curated AI data sets, and the
(17:45):
risk here is called split viewpoisoning.
They're saying that the riskarises because these data sets
often contain data that ishosted on domains.
They may have expired or nolonger actively maintained by
their original owners.
In such cases, anyone whopurchases these expired domains
gains control over the contenthosted on them.
(18:07):
And this situation creates anopportunity for malicious actors
to modify or replace the datathat the curated list points to
potentially introducinginaccurate or misleading
information into the data set.
And that is a great examplethat's represented by.
When you see that a domain hasexpired and goes into the 30 day
grace period, someone comesalong or someone might be
(18:29):
waiting, they may have put theirname down on the waiting list to
purchase that domain, and theycan use that domain's reputation
and that provenance to do somecontent injection later on
because people are stillreferencing that domain.
It's the dangers of the internetthat have not gone away, but
we've just become accustomed toand kind of forgotten over time
(18:50):
that that's, that's a risk that.
That is present.
Then doing things like webscraping that presents its own
set of risks because collectingdata on mass like that, there's
no real quick and easy way toverify that the data you've
captured is accurate.
(19:11):
And the main point that they'retrying to drive home here is
that it's not particularlysophisticated.
This is, these are some basicattacks and whether they are
done by.
Advanced actors who still favorusing really simple stuff, using
really simple techniques toachieve their objectives because
why spend more time or why spendmore money if you have to?
(19:36):
These are things that we need topay attention to.
Coming up, we have theadversarial machine learning
threats.
This is where we're talkingabout the four chan groups or
the Reddit groups where peopleare deliberately trying to
deceive, manipulate, or disruptthe AI system, and the malicious
(19:58):
actors are employing datapoisoning to try and corrupt the
learning process, which iscompromising the integrity of
the training data set.
There are a few mitigationstrategies that you can use to
try and avoid this fromhappening.
There are some algorithms thatcan do some anomaly detection,
(20:19):
so if they start to detect thatsomething's not quite right, it
could be a pirate copy.
There's also data sanitization,which is sanitizing the training
data by looking at usingtechniques such as data
filtering and normalization, andthat is looking for outliers in
the data and it's trying to grabthose high quality results.
(20:42):
And sanitization needs to happenon a regular basis.
And that is prior to and aftereach training session, I guess
we could call it, or when finetuning is occurring or any other
process that might change themodel parameters.
So the model is the, the AIengine or the AI brain itself.
(21:06):
So going through this entiredocument, it is a, a very
interesting read.
I would say that it is probablymore aligned to data scientists
and AI information architects.
It's not really suitable for thestandard consumer.
And the reason I say that isbecause the standard consumer,
(21:29):
like pretty much myself, I ammaking some AI apps and doing
some, some development there.
I'm still using off the shelftools.
I did use an alama build.
To build an AI platform at home.
I kept it really, really simpleand I haven't looked at that for
about 12 months.
(21:49):
And the other tools that I'musing are Clawed and Jet GPT as
my, as the model.
I haven't used Deep Seek, and soa lot of the recommendations in
this report are not reallyrelevant to the way that I'm
using ai, even though I've beenbuilding custom GPTs and chat
bots.
I'm still referencing a modelthat is a commercial model
(22:09):
that's available, and I haven'tstarted to fine tune those
models either.
And so I'm relying on the factthat I'm paying for those
services, that they willactually be performing the risk
mitigation tactics to prevent mymodels or my ability to use
these models from being poisonedor providing junk data.
(22:32):
And it's still the case that Ido.
Get a lot of junk responses, andwhat that comes down to is my
prompts not being specificenough, and it's a little bit
more nuanced than what should betalking on a forensics podcast.
I suppose what needs to happenis if you're finding that the
(22:54):
results that you're getting havebeen poisoned.
That is probably part of abigger problem if you're,
especially if you're running aninternal LLM large language
model in general.
If you're still using commercialLLMs like Claude or Google
Gemini or chat T or one of theothers, try and go back and do
some better prompt engineering.
Be a bit more specific with howyou were doing it.
(23:18):
That is all I've got today fromthe summary of the AI data
security paper.
Best practices for securing dataused to train and operate AI
systems.
And as you've probably seen onLinkedIn, I'm starting to make a
bit more of a, a move towardstalking about AI and while still
having a very strong digitalforensics incident response
(23:39):
focus, I can see that.
The use cases for AI will behuge to help us process data.
I also feel that it's quitenecessary for us to maintain an
AI skillset because threatactors are utilizing AI to
attack us more and more.
The ability for AI to be used toweaponize attacks and to cause
(24:02):
more damage and require moredigital forensics and incident
response.
Is going to be greater than everbefore.
And despite using some automatedor some best practices for
triage techniques and using somegreat digital forensics
acquisition tools, theprocessing of the mountains of
data that we are going to needto do means we must get better
(24:26):
at learning how to use AI appsand the game of cat and mouse
between red and blue teams orthreat actors and defenders.
He's not going away.
And I would encourage everyoneto get into AI as much as you
can and start learning becauseit is not going away.
And the only thing we can do totry and keep defending against
(24:47):
the bad guys is to learn asquickly as they are learning to
attack us.
So I hope this was informativefor you today, and I'll see you
in the next episode.
Bye for now.
Hey everyone.
Welcome back to TLP.
I'm your host, Clint Marsden,and today we're gonna talk about
something that's getting a bitmore airtime and dunno if it's a
bit more scary or if it's justbecoming kind of the norm.
But if you picture this, AIsystems are essentially
everywhere now, right?
(00:22):
They're in our phone, they're inour car security systems,
protecting criticalinfrastructure.
They're everywhere.
All of these systems arelearning AI from some kind of
data.
The AI has not come up with thison its own.
(00:43):
It has like all learning hasbeen given this as a, a data
set.
Of course, if some evil doer,some bond villain.
Has gone and messed with thatdata.
If they've modified it, putthings in there that are
inaccurate.
(01:04):
Then basically this controls howthe AI thinks.
I'm sure you remember a fewyears ago when AI or automated
robots online started to getsome airplay.
I think there was a researcherwho created an AI bot to go and
(01:24):
buy things.
Online and very quickly itstarted to just go and buy drugs
on the dark web.
It was kind of a, kind of an anadvanced shopping bot and it
shows how quickly things canturn without the correct, fine,
fine tuning and without thecorrect guardrails.
(01:47):
And there have been otherinstances of early AI systems
where when they went onto theinternet.
The likes of people from fourchan or the likes of people from
Reddit have decided for onereason or another that they want
to try and take this down andjust fill it with junk and to
fill the training model withjunk data.
(02:09):
Maybe they're anti ai, maybeit's just for the lulls.
Who knows?
But that is what happened.
And most recently we've seen areport that's been created in
collaboration with a lot ofintelligence agencies.
I believe it's been spearheadedby CSA as well as the NSA, the
F-B-I-G-C-H-Q, Australia's a SD.
(02:31):
These are all intelligenceagencies, including other
intelligence agencies fromaround the world.
Was also cool to see somerepresentation from the New
Zealand Cybersecurity Center.
They're now reporting, this isbecoming a lot more prevalent,
the hijacking.
Of AI systems also known aspoisoning, is, is becoming the
(02:52):
new norm.
It is not expensive to perform.
We are talking the cost of, say,a laptop.
In some cases we're looking at,so if a cost of a laptop might
be two and a half thousanddollars, then there are also
cases of this being done forunder$60 or under a hundred
(03:13):
dollars.
As little as a little, as littleas 60.
The training model of of AI islike learning as a child, and
it's like teaching a child howto recognize things.
What we are doing while we'relearning to learn, we look at,
sometimes we look at pictures ofthings, and if we're teaching a
child, uh, or if we're learninganother language, you might
(03:33):
receive an image and you showthem a picture of a cat or a
picture of a dog, and you showedthem this and you say, this is a
dog.
And over time throughrepetition.
That cognitive enhancement, theylearn the difference.
What we're talking about withthis AI poisoning is someone
kind of coming in and, andflipping that around.
(03:56):
The label being this is a catwhen it's actually a dog, and
vice versa.
The technical term here is, isactually called data poisoning.
What we're really doing is,well, we're making a rod for our
own back by by doing this, butwe're just lying to, we're lying
to robots.
Because the AI is in thisimpressionable phase, it doesn't
(04:17):
have any idea that it's beenlied to.
It's, it's developing, it'slearning these capabilities.
It is designed to learn what ithas been taught.
This goes into the memory, andthen unfortunately over time
these issues compound because itis retaining this incorrect
(04:38):
information that it has beentaught.
And then it's making decisionsand delivering output based on
that poisoned information.
The report that CSA havereleased has got six stages
where things can go wrong with,with this data poisoning.
And in a, in a brief summary, itstarts off where, when
(05:02):
organizations are planning thearchitecture and the design of
their AI systems.
And then it finishes at when thesystem is running in production
and it's making real decisionsand it's, it's informing real
decisions made by humans in theloop and humans in the loop, if
you don't know, is just a stepwhere you can run an AI
(05:23):
automation and at some point itstops and waits for someone to a
human to physically verify thatwhat it is proposing to do is
okay.
So this might be useful in a,the con.
I mean, I don't have any contextto give you for something in a
nuclear power plant of anything,anything like that.
But what I'm looking at at themoment is content creation using
(05:47):
ai and before it goes out, Iwanna make sure that it matches
the tone and structure and hasinformation that I would publish
if I was writing it.
And if I need to, I will thenget in there and adjust it.
Accordingly and make sure thatit is delivering the message
that I want.
Because the last thing that I'mtrying to produce is what people
(06:09):
are calling AI slop.
I don't want to produce contentthat's AI slop because it goes
against everything that I'mtrying to do.
I'm trying to produce contentthat is helpful for the reader
or is helpful for the listener.
So that's what Human in the Loopdoes that gives you that, that
kind of last chance before itgoes live.
(06:30):
To verify and make any changesand then you can publish it and
to get around these perceivedproblems.
And I say perceived because Idon't have any physical examples
of them right now, to give you,not to diminish the seriousness
of, of what they are.
The document has provided uswith some frameworks that we can
(06:54):
use.
The first thing is we need toknow where our data's coming
from.
So we're going back to some realfundamentals here.
Like if you download software onthe internet, you see there's an
option to download thecryptographic hash as well.
Just to go into a little bit ofa segue, say you're running
Microsoft have retired this now,but say you're running a red
(07:17):
forest, you're running hardeneddomain controllers in a tiered
environment in a T zeroenvironment, which might contain
your domain controllers andother T zero systems.
Like exchange servers.
Remember those before we went tooffice 3, 6, 5.
Before you install any softwarein this environment, once you've
downloaded the installerpackage, or even if you've
(07:39):
already got the installerpackage, you need to make sure
that it passes the hashingalgorithm to ensure that it
hasn't been modified in transitor has been modified repacked,
loaded with some malware, loadedwith something dodgy in the time
that it's been sitting on yoursystems.
We won't get into, but how doyou make sure that the
(08:00):
cryptographic hash that's beingprovided is correct?
'cause that just creates a bitof a loop and I dunno, honestly,
dunno how to solve that one.
So what they're suggesting isuse a cryptographic hash to
verify the data and you canstill hash the contents of
documents.
It's not only reserved forbinary files, they also talk
(08:20):
about using blockchain tracking.
And if it fails, these checksdon't use it.
This isn't so much of an issue.
If the document is not a hugetome, that will take a long time
to view.
But I recently purchased a bookon AI and red teaming, and
specifically red teaming the AIplatform.
(08:41):
The book is a thousand pages.
It's gonna take me a long timeto get through that book.
It's gonna take me months.
That is where the, using acryptographic hash to verify
that the contents has not beenmodified before, using it as
training data is gonna be quitehelpful.
Second, they're talking againabout maintaining the data
(09:02):
integrity during storage andtransport, and this is, this is
where I kind of disagree in howthey've done this.
It's, this is kind of one point.
Yes.
If we're downloading the contentof the information.
Sure.
We need to verify that thecontents is what we expect it to
(09:22):
be, and then if it's beingstored well before, it's then
used in the training data.
Yes, hash it.
Again, similar to forensicpractices where we might be
running.
Every time the data istransferred or moved or used,
you might decide to run a hashto make sure that it is
maintaining that integrity.
So moving on to number three.
(09:44):
Again, same kind of concepthere.
Employing digital signatures toauthenticate trusted data
revisions, using a CA to ensurethat things are not being
modified in transit.
And I'm just gonna move on tonumber four, which is leverage
trusted infrastructure.
(10:06):
Remember those words?
Zero trust.
If you're still trying to getyour zero trust program up and
running, I understand it's a bitof a beast.
The point here is that they'retalking about providing a secure
enclave for storing your data.
Again, I feel like we are stillon the same point.
Uh, I can't believe that we havefour points of this
(10:28):
recommendation that areessentially bundled into the one
concept.
Keeping it in a secure locationand hashing it and then
verifying it, and then makingsure that it can't be modified
in in transit.
This could be some real strongrecommendations.
This could be from areas of thegovernment that we might not
normally hear from, wherethey're operating in such a
(10:52):
secure environment that theseare the measures that they have
to take.
I kind of feel that these aresome super secure areas.
Or some super securerecommendations that they would
follow, maybe in top secretnetworks.
I've never worked in a topsecret network, so I'm only
guessing, but listening orreading the amount of rigor
(11:14):
that's needed for these types ofscenarios, it seems, it seems
pretty plausible.
Then of course, we're talkingabout user access control,
role-based access control, RAC.
Making sure that only the rightpeople are allowed to access
these things.
That comes down to principle ofleast privilege, something that
(11:36):
I spoke about very, very earlyon in the, one of the, I think
the first four podcast episodeswhen I was doing that NIST
series.
Making sure that only the peoplewho are going to be working with
the system, the, I guess thedigital librarians are the ones
that have the access.
You don't wanna be training theAI again, once you do it, once
(11:57):
you get this first hurdle out ofthe way.
So you wanna make sure thatyou're training it with the
right stuff.
Then of course, encryption,that's a bit of a no brainer
these days.
It's, it's so easy to ensurethat things are encrypted and
that covers not just data atrest, but data in transit.
So data at rest, sitting on thedrive, sitting in a database,
(12:17):
data in transit, going across anetwork, going from cloud hosted
platform where your AI might bestored, even if it's private
cloud.
Or even if it's just across yournetwork, these may not be
problems that you need to dealwith right now, but as the
company expands, these arethings that you will need to
(12:39):
ensure that they're covered.
As this system becomes a bitmore of a mission critical
system for you, it's somethingthat you'll need to have anyway.
And the point is to do this nowso that you're not dealing with
technical debt later.
I've seen technical debt so manytimes and over so many years at
all these differentorganizations.
And I think it's easy to try andavoid doing things right
(13:02):
initially from a place of weneed to get this out.
We're behind on the project.
We want to implement it.
It needs to happen.
Now there might be somedownwards pressure from above to
make things happen.
Maybe there are other reasons.
Competitive advantage.
The sooner we can get this out,the sooner we can take more
market share.
(13:23):
Great.
Let's do that, but let's do itsecurely.
Let's follow these basicfundamental principles right
now.
Hopefully it's why you'relistening to this particular
episode because you wanna knowwhat do we need to do now to
protect us for the future?
Again, number seven, talkingabout storing data securely kind
(13:44):
of done to death by now.
Talking about the use ofcryptographic modules and
encryption.
It's quite interesting actually,because this is a TLP clear.
So you've got TLP, clear green,amber, and red for something
that's A TLP Clear.
It seems to have such a deepresonance with things that may
(14:06):
not be in a TLP clearenvironment or maybe in
documents that you mightgenerally not have access to.
So it's pretty cool that thegovernment is sharing these
types of things with us asprivate citizens.
Moving on to number eight.
Leveraging privacy, preservingtechniques, and then doing
things that will make yourprivacy officer very happy and
(14:28):
very proud of you.
Doing things like data maskingthat is removing sensitive data
with and replacing itinauthentic, but realistic
information basically means whenyou look at the data set, it
still looks normal, but insteadof Clint must, and in the
record, it's.
Tom Jones.
Of course, it needs to be namesthat do not exist in your
(14:51):
customer base to make iteffective.
It's not enough to just move therows around and move someone's
name from row 2000 to row 500.
It needs to be completelyunique.
Number nine is deleting datasecurely, working with physical
hard drives, and then it's timeto dispose of them because they
have failed.
(15:12):
This is important even if thedrive appears to have failed.
Or if you are at end of lifethree years down the line, four
years down the line, and youdecide that it's time to upgrade
these systems, obviously thebest recommendation is to
physically destroy them.
Putting them through a shredderis a common favorite method of
doing that, engaging a thirdparty company.
(15:33):
I used to drill holes in theplatters of hard drives before
we threw them out, and there's alot of discussion on this.
And Rob Lee from Sands.
Talks about it and the, I don'tknow if it's a myth, but there's
the knowledge of an electronmicroscope being used to recover
data.
And this was back in the dayswhen hard drives were only 20
(15:55):
gig.
Uh, now I've heard that thedensity of data is so large that
even an electron microscope isnot gonna help anyone trying.
I guess it depends on yourthreat model.
Uh, if you are worried aboutpeople using an electron
microscope to recover data fromthe drives.
Melt them down, shred them, dowhatever you need to do.
But physical destruction isprobably the best.
(16:18):
Talking about data wipingmethods, the DOD three pass, or
the eight pass, or the 16 passtype wipes, from what I have
read in the research, notnecessary, just a single proper.
Once a once it is wiped, it iswiped.
Not a quick format.
(16:39):
Quick format just markseverything available for
deletion.
That's why it's quick.
The data is still there.
It's not physically there.
Doing a zero of the drive flipsall the bits to zeros, and in
the case that it's already azero, I believe that it flips it
to a one and then back to a zeroto ensure that it's fully gone.
And then lastly, number number10 or step 10 here is to conduct
(17:02):
ongoing data risk assessments.
So using frameworks like theNIST SP 800 dash three R two
haven't heard of the three R twobefore must be a a latest
revision.
The idea here is to evaluate thesecurity landscape, identify
risks, and then prioritize theactions that are relevant to
(17:24):
your organization to minimizebreaches or security incidents.
They then move into somedifferent types of risks and
these risks.
The first one that they'retalking about is a curated web
scale data set.
They make reference to a coupleof curated AI data sets, and the
(17:45):
risk here is called split viewpoisoning.
They're saying that the riskarises because these data sets
often contain data that ishosted on domains.
They may have expired or nolonger actively maintained by
their original owners.
In such cases, anyone whopurchases these expired domains
gains control over the contenthosted on them.
(18:07):
And this situation creates anopportunity for malicious actors
to modify or replace the datathat the curated list points to
potentially introducinginaccurate or misleading
information into the data set.
And that is a great examplethat's represented by.
When you see that a domain hasexpired and goes into the 30 day
grace period, someone comesalong or someone might be
(18:29):
waiting, they may have put theirname down on the waiting list to
purchase that domain, and theycan use that domain's reputation
and that provenance to do somecontent injection later on
because people are stillreferencing that domain.
It's the dangers of the internetthat have not gone away, but
we've just become accustomed toand kind of forgotten over time
(18:50):
that that's, that's a risk that.
That is present.
Then doing things like webscraping that presents its own
set of risks because collectingdata on mass like that, there's
no real quick and easy way toverify that the data you've
captured is accurate.
(19:11):
And the main point that they'retrying to drive home here is
that it's not particularlysophisticated.
This is, these are some basicattacks and whether they are
done by.
Advanced actors who still favorusing really simple stuff, using
really simple techniques toachieve their objectives because
why spend more time or why spendmore money if you have to?
(19:36):
These are things that we need topay attention to.
Coming up, we have theadversarial machine learning
threats.
This is where we're talkingabout the four chan groups or
the Reddit groups where peopleare deliberately trying to
deceive, manipulate, or disruptthe AI system, and the malicious
(19:58):
actors are employing datapoisoning to try and corrupt the
learning process, which iscompromising the integrity of
the training data set.
There are a few mitigationstrategies that you can use to
try and avoid this fromhappening.
There are some algorithms thatcan do some anomaly detection,
(20:19):
so if they start to detect thatsomething's not quite right, it
could be a pirate copy.
There's also data sanitization,which is sanitizing the training
data by looking at usingtechniques such as data
filtering and normalization, andthat is looking for outliers in
the data and it's trying to grabthose high quality results.
(20:42):
And sanitization needs to happenon a regular basis.
And that is prior to and aftereach training session, I guess
we could call it, or when finetuning is occurring or any other
process that might change themodel parameters.
So the model is the, the AIengine or the AI brain itself.
(21:06):
So going through this entiredocument, it is a, a very
interesting read.
I would say that it is probablymore aligned to data scientists
and AI information architects.
It's not really suitable for thestandard consumer.
And the reason I say that isbecause the standard consumer,
(21:29):
like pretty much myself, I ammaking some AI apps and doing
some, some development there.
I'm still using off the shelftools.
I did use an alama build.
To build an AI platform at home.
I kept it really, really simpleand I haven't looked at that for
about 12 months.
(21:49):
And the other tools that I'musing are Clawed and Jet GPT as
my, as the model.
I haven't used Deep Seek, and soa lot of the recommendations in
this report are not reallyrelevant to the way that I'm
using ai, even though I've beenbuilding custom GPTs and chat
bots.
I'm still referencing a modelthat is a commercial model
(22:09):
that's available, and I haven'tstarted to fine tune those
models either.
And so I'm relying on the factthat I'm paying for those
services, that they willactually be performing the risk
mitigation tactics to prevent mymodels or my ability to use
these models from being poisonedor providing junk data.
(22:32):
And it's still the case that Ido.
Get a lot of junk responses, andwhat that comes down to is my
prompts not being specificenough, and it's a little bit
more nuanced than what should betalking on a forensics podcast.
I suppose what needs to happenis if you're finding that the
(22:54):
results that you're getting havebeen poisoned.
That is probably part of abigger problem if you're,
especially if you're running aninternal LLM large language
model in general.
If you're still using commercialLLMs like Claude or Google
Gemini or chat T or one of theothers, try and go back and do
some better prompt engineering.
Be a bit more specific with howyou were doing it.
(23:18):
That is all I've got today fromthe summary of the AI data
security paper.
Best practices for securing dataused to train and operate AI
systems.
And as you've probably seen onLinkedIn, I'm starting to make a
bit more of a, a move towardstalking about AI and while still
having a very strong digitalforensics incident response
(23:39):
focus, I can see that.
The use cases for AI will behuge to help us process data.
I also feel that it's quitenecessary for us to maintain an
AI skillset because threatactors are utilizing AI to
attack us more and more.
The ability for AI to be used toweaponize attacks and to cause
(24:02):
more damage and require moredigital forensics and incident
response.
Is going to be greater than everbefore.
And despite using some automatedor some best practices for
triage techniques and using somegreat digital forensics
acquisition tools, theprocessing of the mountains of
data that we are going to needto do means we must get better
(24:26):
at learning how to use AI appsand the game of cat and mouse
between red and blue teams orthreat actors and defenders.
He's not going away.
And I would encourage everyoneto get into AI as much as you
can and start learning becauseit is not going away.
And the only thing we can do totry and keep defending against
(24:47):
the bad guys is to learn asquickly as they are learning to
attack us.
So I hope this was informativefor you today, and I'll see you
in the next episode.
Bye for now.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
Ridiculous History
History is beautiful, brutal and, often, ridiculous. Join Ben Bowlin and Noel Brown as they dive into some of the weirdest stories from across the span of human civilization in Ridiculous History, a podcast by iHeartRadio.