April 15, 2024 • 31 mins
Welcome to Dev.Sec.Lead, where today we dove deep into the transformative role of Retrieval Augmented Generation, or RAG, in cybersecurity. Joining us was the brilliant Brennan Lodge, who illuminated how this tech can serve as an AI assistant, enhancing accuracy and combating AI hallucinations. Crucially, we touched on RAG's potential to upskill analysts, integrate threat intelligence, and significantly improve decision-making processes by mapping alerts to MITRE ATT&CK techniques. Despite challenges, it promises a more secure future. Stay vigilant, stay informed, and remember, AI is an aid, not a replacement for human ingenuity. Don't miss out—like, subscribe, and bring your friends into the fold of Dev.Sec.Lead for more episodes on cutting-edge cybersecurity!
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(01:19):
Okay. I've got my guest here, Brennan Lodge. Wemet over at the Sunshine Cyber Security
Conference here in Tampa, Florida, and he talked about something pretty awesome,and we wanted to bring him on the show today to talk about a little
bit regarding retrieval augmented generationand AI and how that impacts cybersecurity.
(01:41):
So, hey, Brennan. Thanks for coming on to the show today. Yeah.
Pleasure. Thanks, Austin. Yeah. Yeah. So AI. Right? Everybody'stalking about AI. Right? And it's like this supposedly
secret sauce to a lot of different things. Everybody knows what ChatGPTis. Right? And they're all talking
about, okay, this is generative AI, and it's able to giveme answers based off of a huge dataset. But
(02:07):
what makes retrieval augmented generationAI different than what we see at Chat GPT? Yeah. So the the
hype is real. Right? We're in the hype cycle of AI,and you've got the extremes of AI saving
the world or AI ending the world. And in between, we'vegot cybersecurity, more so hopefully on the safe side.
(02:30):
Red team, blue team. There you go. But yeah. So RAG, I wasin a r and d team at my last bank that I
worked at. And, of course, that came to ourteam, you know, from the top down of how are we gonna
use AI? Is it hype? Are there some realisticimplementations that we can use it to to help us? And this is,
(02:52):
like, the theme that I've been, you know, preaching over and over of, youknow, the realistic implications of it, which is
AI for us is a guide. AndI I don't see it as a full automated guard
just yet, given some of the, you know, white papers andstuff that that I've read. So getting
(03:14):
closer to to Rag. Right? So I started with what isgoing wrong with AI? Why
is it bad or not accurate or canfail? And the big thing with
that was on the hallucinations ofbad output. And we're all we've all seen it, right, on
(03:37):
ChatChippet or some other generative AI offering.
And I started, you know, reading about retrieval augmentedgeneration as a combatant, really, or an assistant,
really, to just the raw LLM or generativeAI response. And, yeah, it just went down the rabbit
hole and got lucky. Right? Rolling the dice onwhat research or technique or tactic with generative AI should
(04:04):
I go down. And it's been really powerful, that being aretrieval augmented generation. I think that one of
the things that I see with AI, especially with using chat gpt,and this is where I think RAG is really beneficial, is
that with regular chat GPT,generative AI, you don't get to see where the sources are.
(04:27):
Yeah. And that's really important because it's okay. Is this garbage, or is thisactually fact? And with Rag, we're able to
take a data set that's very I guess, you put PDFsin the knowledge base, and then it's able to cite those PDFs. And that's the
extent of how I've used it. I'm really interested in how wecan look at it from a cyber security standpoint. How
(04:50):
does that work? You've got some experience with the SOC, and you got experience.
We were we've been talking about GRC stuff. What what are you thinking? How doesit how does RAG enhance cybersecurity efforts?
Yeah. So maybe we paint the picture, use a analogy I've beenusing for RAG to set the stage. So with
retrieval augmented generation, there's like a trifecta ofintegration points. Right? So you've got
(05:16):
a sentence embedding model, a vector database,and an LLM. And you have a prompt
and an answer. Right? Prompt at the beginning, answer at theend. So you have a sentence embedding model,
which translates your English or whatever languageinto machine language or embeddings
(05:38):
within a vector database. So you're going toparse out those PDFs in a schema
and upsert them into the vector database. So nowand vector databases on their own are an awesome tech, but we can talk
about them later, of just semantic search.
Right? What the embeddings do, given thetext that you observed into that vector database is kinda
(06:06):
similarities. So the similarities I think of your reading ofa a story. I've got a 3 year old daughter, and we talk about kings
and queens and prince and princesses. Of course, she's a princess.
Right? But how do you make the similarities of she,queen, princess, daughter, son,
prince. Right? That's the power of the vector database inthe embeddings that translate based off of the similarity
(06:31):
searches that you can get. So we've got thatbuilt out within the reg, and now we've got an LLM.
So the LLM can do that translationof answering that question based off of what the LOM
was trained. And with that trifecta, you cannow get a cross reference based off of what
(06:55):
was in that query or question and pointto the document that was parsed
and inserted into the vector database. Andalso, you've got the l m response. So understands that
content, that question, and on its own withthe power of that cross reference can provide a really
(07:17):
powerful answer. So you're you still couldget the hallucinations, but you're also going to get a cross
reference. So you can get a link to that PDFor a link that within the section or
the paragraph of that PDF so that it'llprove, hey, this LLM answer and also the cross
(07:39):
reference answer. And if you wanna dig more, hey. Here's the thecitation as to where that answer is closely, located
in in that document. And I I know that we're gonna talkabout PDFs, but, like, from a I guess, from a context
of an analyst position and usingthat in regards to, like let's just
(08:01):
talk about something that hits near and dear with a lot of analysts hearts,like log parsing and log understanding logs. How do you
see RAG as a a tool to help analysts do theirjob better? Yeah. So I see it really
well on the upskilling. Right?
Especially if you're able to integrate threat intelligence.
(08:23):
We're all familiar with those PDF reports on Cozy Bear,for example. As a junior analyst, it just sounds like a funny
name to everybody else. Right? But what kindof context can we get out of reg, which is given the
TTPs of cozy bear that may havebeen flagged, within the logs or within the alerts.
(08:46):
Now we can query and ask, like, alright. Given Cozy Bear andgiven this alert I've seen, tell me more. Tell me more
context to help me, understand why I shouldcare about this investigation. And it can
point back to that PDF or most recent reporton Cozy Bear. So that's like the the threat intel angle,
(09:09):
especially for the analysts. Yeah. I think that would be reallyinteresting to see how, like, you would have the external information
as a database, and then you could addmore contacts with internal information and be able to relate the
2. Maybe that's something that will come in the futureor is probably already happening. I just don't know. Yes.
(09:31):
It it is happening. Yeah. So one of the use cases real quick is onMITRE mapping, An open source project that I'm working on
with that vector database approach of, say, I havean alert or a log or an event or,
right, some threat intel. Hey. Let's map this to a MITRE ATT andCK. Right? Now we all know we can't get those 1 to 1
(09:53):
matches or what we can get one to many we'll rankthis match for the MITRE ATT and CK technique. Again,
providing that that context, that know how andinstitutional knowledge that a junior analyst may
not have. And it also helps downstream with that contextand and mapping for the grand scheme of all of our detections, all of our
(10:14):
alerts and Mhmm. What's been going on within our logs.
Yeah. It seems if we get it to work really well, that it would helpanalysts make their decisions faster or be able to provide information
faster to the higher ups and their managers to make decisions.
But I but I think we're a little bit prematurein saying that it's gonna happen right away. I think it's probably gonna happen in
(10:37):
the next, I don't know, 5 years, 10 years, possibly.
But what are some of the challenges with integratingRAG into existing cybersecurity infrastructures?
Yeah. So, you know, it comes down to to cost.
Right? Yeah. Always money. Right? Always ROI. Yeah. So let'sstart with the money. GPUs are expensive. Right? I think later down the
(11:01):
road, most tech will get cheaper. But fornow, on that trifecta, a lot of it can be
open source, which is a beautiful thing of especially on the opensource, models in in that community. And
but in order to process this, it takes some horsepower.
Right? So GPUs are expensive. And I would try andset, like, a benchmark with it. So
(11:28):
my benchmark was I needed I wanted to get a response within 10seconds based off of a query and keep
it under $500 a a month. And we all know EC2 and AWS in the cloud, it's not so transparent with the
the cost. So being sure you're able to to manage and and track thataccordingly. After some efforts, was able to to
(11:50):
get it at that benchmark and set somestandards of and, of course, the the limitations given the
resource constraints with it. But, yeah, and then theother thing is is privacy. So we don't want our
intellectual property out the window, especiallyif we make requests to some of these API services that are offering
(12:13):
these that OpenAI's of the the world. You make aquery to OpenAI that's going over the Internet.
We've seen some of the code show up or proprietarycode and questions and it and answers come back,
and that's scary. The alternative and with myoffering, I've really pushed through to have this as an on
(12:35):
prem solution Mhmm. And bring this into aVPC or your on prem, and you can
stand this up and not have to use the open AIs of theworld, to make those queries and customize.
Right? Further fine tune and train that LLMto understand some of the custom terminology that you
(12:58):
may ask of it compared to maybe OpenAI AI orsome other LLM out there may not have. So that's another
beautiful thing, but also a consideration. So when you saywhen you're saying LLM, right, and we're just, like, considering OpenAI
and you have Grok, and you have all these othertools that are out there. Are you saying that's would it be better for
(13:22):
companies that want or organizations or or people in generalto utilize a proprietary like, their own LLM
to, do this kind of work because youjust don't wanna use anything that's out in public.
Yeah. So most of these models, if not all, aresomewhat black box. Right? Even the data scientists
(13:45):
that are building them can't really explain onthese neural networks or how all they're reacting or
acting or providing such awesome accuracy.
Right? There are some open sourcemodels that are trying to get there with that transparency. GPT for
all is one of my favorites and the one I've been testing and playingaround with. And they provide
(14:11):
the a really cool data visualization toolto visualize what data was used
to train the model. Right? Sothat can help. And back to your
question on where we're going, I thinkthe Google's, Microsoft's, and everybody creating a a
(14:34):
model out there is trying to understand the entire world or theentire Internet. But we'll slowly start to see
the opposite effect of domain specific models.
And that with training on specifictypes of of data, cybersecurity, for example. If you take a look in
the the Hugging Face inventory, we're already getting there. There isa cybersecurity model called Lily that I've tested. It's
(15:01):
okay. But going back to that transparency, it doesn'tprovide the data used to train the model, and that can
help with understanding, like, any biases or any particularsubject matters, for example, that specialize for that
model. So back to the GPT for all, I, you know,searched through a few just like a quick search and index of the data
(15:23):
used for the model. I found 1, a good one, called wizardthat had some data on phishing. Right? Not
phish the the band, but phishing the cybersecurity technique.
And it was like, okay. But it's got some cybersecurity domain knowledgeinherent in it. And lo and behold, it had insecure
code in there that was identified, that was trained against it.
(15:46):
And some other subject matters of application security,intrusion detection. So I'm like, okay. Found something good.
And yeah. And I've been using that off the shelfwithin my reg for just playing around with the
research. So, yeah, I think that's where the industry is going, andwe just need to be more transparent with what data we're using
(16:08):
to train our models. No. That's awesome. Thanks. So let'stalk about the the implications.
I can't speak today. Implications. The implicationsof RAG and how we can utilize
this as a benefit for cybersecurity organizationsfor our up and coming leaders. Right? So what should cybersecurity
(16:31):
leaders know about the scalabilityof RAG solutions, RAG AI solutions?
Yeah. So scalability. Right? It'snot just about handling more data or more
queries. It's also how well we can integrateit to our existing infrastructures, our existing
(16:53):
tools, our platforms, our our workflows. Andthat's where I think the devil in the details it is and getting the
the gold findings out of RAG. And ittakes quite a bit, of tuning, right, to to get
there. Again, not a guard, buta guide. So I think we'll start seeing, like, reg. It's just
(17:16):
like a hybrid type of system of the integration,knowledge, share, quick searching and querying,
and maybe that MITRE mapping or even helping withscoring on some of the detections on priorities. And
then the cost. Right? So we talked about how well this can scale.
We can't make a million requests to therag and expect it to be well, and, of course,
(17:43):
garbage in, garbage out, right, with the the data. If you're putting, youknow, unstructured data of of garbage in there,
you're going to get bad results. So this is whatI've been living in the last 10 or so years of building data science teams
within cybersecurity departments of, hey. We want AI or wewant ML, Brennan, but the foundation of our
(18:06):
data infrastructure is crap. Right? So we need to start fromthere and building doing data engineering,
building out and stabilizing those pipelines, and thenadding in some ETL, extract, transform,
load so it's clean. And further downstream,you're gonna get better results that are efficient and maybe
(18:28):
even real time. But you've gotta deliver on that firstin order to get to the fancy AI capabilities.
No. That's great. So with the ragtools and their interaction with other AI systems
currently used in cybersecurity, how do we usethem with, like, machine learning models or neural
(18:51):
networks? Yeah. Great question. We're gonna leadin back to that, like, hybrid approach. Mhmm.
Line decision making feedback loops is is another importantone. Right? So let's jump into the feedback loops, which is
fun and a ML tactic within itself. Buton the use case there, we get our detections,
(19:15):
and those come downstream into our SIEM. How weevaluate those detections are on, like, the data science
confusion matrix, and it's not that confusing. Right? We're all familiar with theseterms, which is false positives, false negatives, true positives,
true negatives. Right? So scaling, yourdetections first into those categories and understanding,
(19:37):
like, how well we're doing. Right? We don't want to overwhelmour analysts. Right? Hence, false positives. Mhmm. And we don't want to
miss anything, hence, false negatives. But how do you balance them?
Right? Feedback loops. Just a thumbs up, thumbs down canhelp at the end when an analyst is closing that
investigation to say, hey. This was a a true positive,or this was a a false positive. Right? And then
(20:04):
tossing that back into the loop of your machine learningand training can then help get better accurate more
accurate results further downstream.
Right? So that helps with the full ecosystem, ifyou will, on on ML.
So with leaders coming into the scene and leaders thatare already in their positions, we all
(20:30):
know that artificial intelligence and generativeAI already has some issues. Right? We know that there are
specific attacks and specificvulnerabilities that are inherent with these AI systems.
Is it any different with RAG?
No. And it's no different really from thetraditional security approach. Right? So with
(20:58):
RAG, because it can be like a achatbot type of application, there are other applications.
They're gonna be vulnerable to prompt injection, data privacyleakage, biases, unfairness with the the
results. Prompt injection attacks has haswarranted a new career for prompt engineers who can
(21:19):
come up with the the best questions and answers. Probably really good attrivia. But on the prompt injection attacks, it
it's similar to a SQL injection attack, similar like crosssite scripting. So taking that same approach
on the manipulation of that input and setting up someguardrails on, hey. We're not going to accept this type
(21:42):
of of input. Yeah. It becomes labor intensive,but, yeah, it's a similar tactic and techniques that we've had to
use to defend ourselves against, you know, the previously mentioned attacks.
Right? And then data privacy leakage. Hey. Let's set theguardrails. No PII, no credit card number,
Social Security numbers, etcetera. We're we're going to use in ourdata, and we're not going to allow that to
(22:07):
be as a as an answer within our data.
You get into some of the more complicated ones on the modelrobustness and the adversarial attacks, or they're
asking questions to try and get to the intellectual propertyor the model parameters. But similar approach, the prompt
engineering to enter those guardrails onavoiding those types of exploits. So
(22:33):
when you say guardrails, I know you mentioned this, a bunch of times, but,like, when I'm thinking of guardrails, it's stopping us from falling over the
edge. But, like, technically, what is that technically, really? Whatdoes what in the world does that mean? Yeah. It's
traditional stuff. Right? So a guardrail of ACLs.
Right? Access control list. Who can get access tothe the model parameters? Who can ask
(22:59):
or be prompting? And then justtaking logs on what questions are being
asked and investigating those logs. Right? Like,typical detection type of, tactics here.
The similar guardrails that that we have for ourtechnical infrastructure can then be gleaned with generative
(23:21):
AI applications. It sounds like there's another jobthat's gonna need to be developed of, like, investigate you said investigating
who's prompting what. That's gonna be interesting. Right. I try to putthat all together. Okay. So as a leader
in this field, how how do you stay updated with theserapid technological advances in AI and cybersecurity?
(23:44):
Yeah. Yeah. It's white papers. It's conferences. It'stalking with folks, listening to your podcast,
Ray Wilson. Mhmm. And just seeing what's outthere. Right? And being realistic and testing and just
trying stuff out, experimenting. Right? Drop some modelsinto your lab and see how far you can take
(24:07):
it. Getting up to speed with RAG was quite difficult. Afun summer project for me, testing out some of the different c p
or GPUs rather. Arm, for example, just to help youguys out with getting up and and riding. ARM GPUs are
really tough to customize and get the drivers workingand such. And to why NVIDIA is doing so
(24:30):
well, especially with their stock price, is the the ease ofuse and their flexibility to work with
developers and just somewhat work, the box.
So CUDA is their development driverpackage, and you can configure and tweak how
many GPUs or or resources on the back endthere to toggle with. But it's fun to the tinkering
(24:55):
around with it and trying out these implementationsand checking out stuff, like, on Reddit or on on
Hugging Face, which is turned into the GitHub formachine learning models, is a good place to research
and and find out what's going on. So what Reddit subsubreddits do you recommend? There's a machine learning
(25:17):
and language technology one. Oh, there's also I think there's onefor reg. There's one I was playing around with COBOL
CPP. So it's k with for COBOL, And that wasa fun application to help con with the configuration
in back end. But, yeah, mileage may varywith what type of subreddits they're on. And, yeah,
(25:41):
that's why I'll just keep it that keep it there. Yeah. No. I totally getthat. You just don't know what you're gonna get. Caution.
That's funny. Alright. So when we're talking about AIdriven technologies, I know that you had mentioned a few things that you've been
working on. But maybe just, like, do you have any kind of listof skills that up and
(26:02):
coming folks need to know? Because, technologyis changing so fast. What do you what are you seeing? You're in
startup world, so you see a lot of you see a lot of stuff thatare innovate innovative in general. So what do you what should
we be paying attention to now? Yeah. So inorder to get up and and running, just start playing
(26:24):
around with Python. It's a universal tool forall data scientists. And then most data scientists have been living
in Jupyter Notebooks. So JupyterLab is a goodway to get up and and running. And most
small models and tools and LMs out thereand the integration or orchestration through
(26:46):
a package called, like, Langchain all works with with Python.
Right? So that's the the beautiful thing. The white papers out there, the Reddits ofthe world, on the enterprise side.
Databricks is pretty good. They've got some really good blogs. I use the ragone to get up and and running. Snowflake is pretty good
out there. And then, yeah, just starting to think,of use cases. Right? So with Rag, I
(27:12):
presented at colonel last week, which was reallygreat conference and technically focused. I'll go mini
Defcon. Look at some really good ideas on the red teamersand their pen test reports. Right? Dropping them into to
reg. Now securing them down, locking them down, puttingsome apples on it on who can get access. But at
(27:34):
the end of understanding the the vulnerabilities orwhat the pen tester took in order to exploit a particular
system is, like, your library. Right? Your inventory that you canreference back and forth or get some new creative ideas amongst the
red team by sharing that knowledge. So that was another cool use caseI didn't think of. And then on the GRC side, starting to
(27:57):
see and actually build out myself some of the use cases onjust data privacy laws, regulatory frameworks out there
that are very cumbersome, very long, lengthy,right, PCI DSS, splitting them up, tossing them in
the the vector database, and asking questions. Also,tossing in your own policy documents into the vector database and
(28:20):
then interfacing with it to compare and contrast, like a gapanalysis of Mhmm. Hey. We wanna do business
in Iceland. Here's our policies. Here's Iceland'sdata privacy laws and regulations. What do
we need to do? You'll get a a list of of things that, you know,didn't match on that those gaps And, hey, write me
(28:42):
a new policy that makes me abide by Iceland's. Right? NowI've got a new policy. It's integrated, customized based off of my own
policy. Of course, you gotta go out there and implement it. Right? Can'tjust write it down, but that's another really good use case that that
I've seen. Just gotta make sure that it's not we'renot looking at AI as the easy button because when like, I live in GRC
(29:04):
world. Oh, yeah. We're just gonna we're just gonna use chatcheappt. We're gonnawrite a we're gonna write all of our policies for SOC 2, and we're gonna
be good. And that's the beautiful thing. I was just humans in the loop.
Right? They're gonna need to interpret. They're going to need to bethat last bastion of, hey, is this
bad? And offering up that insightand logistic reasoning that maybe the AI model
(29:31):
doesn't have just yet. So again,a guide and not a particular guard or full stop
fully automated system. Human needs to be in the loopcontinuously. Yep. Completely agree.
Brennan, thanks so much for your time today. Really appreciate you coming on andtalking to us a little bit and making minds explode
(29:54):
like like mine. It's always nice to be, talking to tosomebody that understands what's going on, especially with AI.
So, thank you again. My pleasure. Thanks all. Goodluck out there, everybody. Hope you liked this
episode of DevSec Lead. We'll have other ones coming up in thenext few weeks. Really appreciate you. And
(30:17):
if you could pass along the message and get your friends tosubscribe and we'll keep on learning together. Hope you all have a good one
and talk to you soon. See you.
Okay. I've got my guest here, Brennan Lodge. Wemet over at the Sunshine Cyber Security
Conference here in Tampa, Florida, and he talked about something pretty awesome,and we wanted to bring him on the show today to talk about a little
bit regarding retrieval augmented generationand AI and how that impacts cybersecurity.
(01:41):
So, hey, Brennan. Thanks for coming on to the show today. Yeah.
Pleasure. Thanks, Austin. Yeah. Yeah. So AI. Right? Everybody'stalking about AI. Right? And it's like this supposedly
secret sauce to a lot of different things. Everybody knows what ChatGPTis. Right? And they're all talking
about, okay, this is generative AI, and it's able to giveme answers based off of a huge dataset. But
(02:07):
what makes retrieval augmented generationAI different than what we see at Chat GPT? Yeah. So the the
hype is real. Right? We're in the hype cycle of AI,and you've got the extremes of AI saving
the world or AI ending the world. And in between, we'vegot cybersecurity, more so hopefully on the safe side.
(02:30):
Red team, blue team. There you go. But yeah. So RAG, I wasin a r and d team at my last bank that I
worked at. And, of course, that came to ourteam, you know, from the top down of how are we gonna
use AI? Is it hype? Are there some realisticimplementations that we can use it to to help us? And this is,
(02:52):
like, the theme that I've been, you know, preaching over and over of, youknow, the realistic implications of it, which is
AI for us is a guide. AndI I don't see it as a full automated guard
just yet, given some of the, you know, white papers andstuff that that I've read. So getting
(03:14):
closer to to Rag. Right? So I started with what isgoing wrong with AI? Why
is it bad or not accurate or canfail? And the big thing with
that was on the hallucinations ofbad output. And we're all we've all seen it, right, on
(03:37):
ChatChippet or some other generative AI offering.
And I started, you know, reading about retrieval augmentedgeneration as a combatant, really, or an assistant,
really, to just the raw LLM or generativeAI response. And, yeah, it just went down the rabbit
hole and got lucky. Right? Rolling the dice onwhat research or technique or tactic with generative AI should
(04:04):
I go down. And it's been really powerful, that being aretrieval augmented generation. I think that one of
the things that I see with AI, especially with using chat gpt,and this is where I think RAG is really beneficial, is
that with regular chat GPT,generative AI, you don't get to see where the sources are.
(04:27):
Yeah. And that's really important because it's okay. Is this garbage, or is thisactually fact? And with Rag, we're able to
take a data set that's very I guess, you put PDFsin the knowledge base, and then it's able to cite those PDFs. And that's the
extent of how I've used it. I'm really interested in how wecan look at it from a cyber security standpoint. How
(04:50):
does that work? You've got some experience with the SOC, and you got experience.
We were we've been talking about GRC stuff. What what are you thinking? How doesit how does RAG enhance cybersecurity efforts?
Yeah. So maybe we paint the picture, use a analogy I've beenusing for RAG to set the stage. So with
retrieval augmented generation, there's like a trifecta ofintegration points. Right? So you've got
(05:16):
a sentence embedding model, a vector database,and an LLM. And you have a prompt
and an answer. Right? Prompt at the beginning, answer at theend. So you have a sentence embedding model,
which translates your English or whatever languageinto machine language or embeddings
(05:38):
within a vector database. So you're going toparse out those PDFs in a schema
and upsert them into the vector database. So nowand vector databases on their own are an awesome tech, but we can talk
about them later, of just semantic search.
Right? What the embeddings do, given thetext that you observed into that vector database is kinda
(06:06):
similarities. So the similarities I think of your reading ofa a story. I've got a 3 year old daughter, and we talk about kings
and queens and prince and princesses. Of course, she's a princess.
Right? But how do you make the similarities of she,queen, princess, daughter, son,
prince. Right? That's the power of the vector database inthe embeddings that translate based off of the similarity
(06:31):
searches that you can get. So we've got thatbuilt out within the reg, and now we've got an LLM.
So the LLM can do that translationof answering that question based off of what the LOM
was trained. And with that trifecta, you cannow get a cross reference based off of what
(06:55):
was in that query or question and pointto the document that was parsed
and inserted into the vector database. Andalso, you've got the l m response. So understands that
content, that question, and on its own withthe power of that cross reference can provide a really
(07:17):
powerful answer. So you're you still couldget the hallucinations, but you're also going to get a cross
reference. So you can get a link to that PDFor a link that within the section or
the paragraph of that PDF so that it'llprove, hey, this LLM answer and also the cross
(07:39):
reference answer. And if you wanna dig more, hey. Here's the thecitation as to where that answer is closely, located
in in that document. And I I know that we're gonna talkabout PDFs, but, like, from a I guess, from a context
of an analyst position and usingthat in regards to, like let's just
(08:01):
talk about something that hits near and dear with a lot of analysts hearts,like log parsing and log understanding logs. How do you
see RAG as a a tool to help analysts do theirjob better? Yeah. So I see it really
well on the upskilling. Right?
Especially if you're able to integrate threat intelligence.
(08:23):
We're all familiar with those PDF reports on Cozy Bear,for example. As a junior analyst, it just sounds like a funny
name to everybody else. Right? But what kindof context can we get out of reg, which is given the
TTPs of cozy bear that may havebeen flagged, within the logs or within the alerts.
(08:46):
Now we can query and ask, like, alright. Given Cozy Bear andgiven this alert I've seen, tell me more. Tell me more
context to help me, understand why I shouldcare about this investigation. And it can
point back to that PDF or most recent reporton Cozy Bear. So that's like the the threat intel angle,
(09:09):
especially for the analysts. Yeah. I think that would be reallyinteresting to see how, like, you would have the external information
as a database, and then you could addmore contacts with internal information and be able to relate the
2. Maybe that's something that will come in the futureor is probably already happening. I just don't know. Yes.
(09:31):
It it is happening. Yeah. So one of the use cases real quick is onMITRE mapping, An open source project that I'm working on
with that vector database approach of, say, I havean alert or a log or an event or,
right, some threat intel. Hey. Let's map this to a MITRE ATT andCK. Right? Now we all know we can't get those 1 to 1
(09:53):
matches or what we can get one to many we'll rankthis match for the MITRE ATT and CK technique. Again,
providing that that context, that know how andinstitutional knowledge that a junior analyst may
not have. And it also helps downstream with that contextand and mapping for the grand scheme of all of our detections, all of our
(10:14):
alerts and Mhmm. What's been going on within our logs.
Yeah. It seems if we get it to work really well, that it would helpanalysts make their decisions faster or be able to provide information
faster to the higher ups and their managers to make decisions.
But I but I think we're a little bit prematurein saying that it's gonna happen right away. I think it's probably gonna happen in
(10:37):
the next, I don't know, 5 years, 10 years, possibly.
But what are some of the challenges with integratingRAG into existing cybersecurity infrastructures?
Yeah. So, you know, it comes down to to cost.
Right? Yeah. Always money. Right? Always ROI. Yeah. So let'sstart with the money. GPUs are expensive. Right? I think later down the
(11:01):
road, most tech will get cheaper. But fornow, on that trifecta, a lot of it can be
open source, which is a beautiful thing of especially on the opensource, models in in that community. And
but in order to process this, it takes some horsepower.
Right? So GPUs are expensive. And I would try andset, like, a benchmark with it. So
(11:28):
my benchmark was I needed I wanted to get a response within 10seconds based off of a query and keep
it under $500 a a month. And we all know EC2 and AWS in the cloud, it's not so transparent with the
the cost. So being sure you're able to to manage and and track thataccordingly. After some efforts, was able to to
(11:50):
get it at that benchmark and set somestandards of and, of course, the the limitations given the
resource constraints with it. But, yeah, and then theother thing is is privacy. So we don't want our
intellectual property out the window, especiallyif we make requests to some of these API services that are offering
(12:13):
these that OpenAI's of the the world. You make aquery to OpenAI that's going over the Internet.
We've seen some of the code show up or proprietarycode and questions and it and answers come back,
and that's scary. The alternative and with myoffering, I've really pushed through to have this as an on
(12:35):
prem solution Mhmm. And bring this into aVPC or your on prem, and you can
stand this up and not have to use the open AIs of theworld, to make those queries and customize.
Right? Further fine tune and train that LLMto understand some of the custom terminology that you
(12:58):
may ask of it compared to maybe OpenAI AI orsome other LLM out there may not have. So that's another
beautiful thing, but also a consideration. So when you saywhen you're saying LLM, right, and we're just, like, considering OpenAI
and you have Grok, and you have all these othertools that are out there. Are you saying that's would it be better for
(13:22):
companies that want or organizations or or people in generalto utilize a proprietary like, their own LLM
to, do this kind of work because youjust don't wanna use anything that's out in public.
Yeah. So most of these models, if not all, aresomewhat black box. Right? Even the data scientists
(13:45):
that are building them can't really explain onthese neural networks or how all they're reacting or
acting or providing such awesome accuracy.
Right? There are some open sourcemodels that are trying to get there with that transparency. GPT for
all is one of my favorites and the one I've been testing and playingaround with. And they provide
(14:11):
the a really cool data visualization toolto visualize what data was used
to train the model. Right? Sothat can help. And back to your
question on where we're going, I thinkthe Google's, Microsoft's, and everybody creating a a
(14:34):
model out there is trying to understand the entire world or theentire Internet. But we'll slowly start to see
the opposite effect of domain specific models.
And that with training on specifictypes of of data, cybersecurity, for example. If you take a look in
the the Hugging Face inventory, we're already getting there. There isa cybersecurity model called Lily that I've tested. It's
(15:01):
okay. But going back to that transparency, it doesn'tprovide the data used to train the model, and that can
help with understanding, like, any biases or any particularsubject matters, for example, that specialize for that
model. So back to the GPT for all, I, you know,searched through a few just like a quick search and index of the data
(15:23):
used for the model. I found 1, a good one, called wizardthat had some data on phishing. Right? Not
phish the the band, but phishing the cybersecurity technique.
And it was like, okay. But it's got some cybersecurity domain knowledgeinherent in it. And lo and behold, it had insecure
code in there that was identified, that was trained against it.
(15:46):
And some other subject matters of application security,intrusion detection. So I'm like, okay. Found something good.
And yeah. And I've been using that off the shelfwithin my reg for just playing around with the
research. So, yeah, I think that's where the industry is going, andwe just need to be more transparent with what data we're using
(16:08):
to train our models. No. That's awesome. Thanks. So let'stalk about the the implications.
I can't speak today. Implications. The implicationsof RAG and how we can utilize
this as a benefit for cybersecurity organizationsfor our up and coming leaders. Right? So what should cybersecurity
(16:31):
leaders know about the scalabilityof RAG solutions, RAG AI solutions?
Yeah. So scalability. Right? It'snot just about handling more data or more
queries. It's also how well we can integrateit to our existing infrastructures, our existing
(16:53):
tools, our platforms, our our workflows. Andthat's where I think the devil in the details it is and getting the
the gold findings out of RAG. And ittakes quite a bit, of tuning, right, to to get
there. Again, not a guard, buta guide. So I think we'll start seeing, like, reg. It's just
(17:16):
like a hybrid type of system of the integration,knowledge, share, quick searching and querying,
and maybe that MITRE mapping or even helping withscoring on some of the detections on priorities. And
then the cost. Right? So we talked about how well this can scale.
We can't make a million requests to therag and expect it to be well, and, of course,
(17:43):
garbage in, garbage out, right, with the the data. If you're putting, youknow, unstructured data of of garbage in there,
you're going to get bad results. So this is whatI've been living in the last 10 or so years of building data science teams
within cybersecurity departments of, hey. We want AI or wewant ML, Brennan, but the foundation of our
(18:06):
data infrastructure is crap. Right? So we need to start fromthere and building doing data engineering,
building out and stabilizing those pipelines, and thenadding in some ETL, extract, transform,
load so it's clean. And further downstream,you're gonna get better results that are efficient and maybe
(18:28):
even real time. But you've gotta deliver on that firstin order to get to the fancy AI capabilities.
No. That's great. So with the ragtools and their interaction with other AI systems
currently used in cybersecurity, how do we usethem with, like, machine learning models or neural
(18:51):
networks? Yeah. Great question. We're gonna leadin back to that, like, hybrid approach. Mhmm.
Line decision making feedback loops is is another importantone. Right? So let's jump into the feedback loops, which is
fun and a ML tactic within itself. Buton the use case there, we get our detections,
(19:15):
and those come downstream into our SIEM. How weevaluate those detections are on, like, the data science
confusion matrix, and it's not that confusing. Right? We're all familiar with theseterms, which is false positives, false negatives, true positives,
true negatives. Right? So scaling, yourdetections first into those categories and understanding,
(19:37):
like, how well we're doing. Right? We don't want to overwhelmour analysts. Right? Hence, false positives. Mhmm. And we don't want to
miss anything, hence, false negatives. But how do you balance them?
Right? Feedback loops. Just a thumbs up, thumbs down canhelp at the end when an analyst is closing that
investigation to say, hey. This was a a true positive,or this was a a false positive. Right? And then
(20:04):
tossing that back into the loop of your machine learningand training can then help get better accurate more
accurate results further downstream.
Right? So that helps with the full ecosystem, ifyou will, on on ML.
So with leaders coming into the scene and leaders thatare already in their positions, we all
(20:30):
know that artificial intelligence and generativeAI already has some issues. Right? We know that there are
specific attacks and specificvulnerabilities that are inherent with these AI systems.
Is it any different with RAG?
No. And it's no different really from thetraditional security approach. Right? So with
(20:58):
RAG, because it can be like a achatbot type of application, there are other applications.
They're gonna be vulnerable to prompt injection, data privacyleakage, biases, unfairness with the the
results. Prompt injection attacks has haswarranted a new career for prompt engineers who can
(21:19):
come up with the the best questions and answers. Probably really good attrivia. But on the prompt injection attacks, it
it's similar to a SQL injection attack, similar like crosssite scripting. So taking that same approach
on the manipulation of that input and setting up someguardrails on, hey. We're not going to accept this type
(21:42):
of of input. Yeah. It becomes labor intensive,but, yeah, it's a similar tactic and techniques that we've had to
use to defend ourselves against, you know, the previously mentioned attacks.
Right? And then data privacy leakage. Hey. Let's set theguardrails. No PII, no credit card number,
Social Security numbers, etcetera. We're we're going to use in ourdata, and we're not going to allow that to
(22:07):
be as a as an answer within our data.
You get into some of the more complicated ones on the modelrobustness and the adversarial attacks, or they're
asking questions to try and get to the intellectual propertyor the model parameters. But similar approach, the prompt
engineering to enter those guardrails onavoiding those types of exploits. So
(22:33):
when you say guardrails, I know you mentioned this, a bunch of times, but,like, when I'm thinking of guardrails, it's stopping us from falling over the
edge. But, like, technically, what is that technically, really? Whatdoes what in the world does that mean? Yeah. It's
traditional stuff. Right? So a guardrail of ACLs.
Right? Access control list. Who can get access tothe the model parameters? Who can ask
(22:59):
or be prompting? And then justtaking logs on what questions are being
asked and investigating those logs. Right? Like,typical detection type of, tactics here.
The similar guardrails that that we have for ourtechnical infrastructure can then be gleaned with generative
(23:21):
AI applications. It sounds like there's another jobthat's gonna need to be developed of, like, investigate you said investigating
who's prompting what. That's gonna be interesting. Right. I try to putthat all together. Okay. So as a leader
in this field, how how do you stay updated with theserapid technological advances in AI and cybersecurity?
(23:44):
Yeah. Yeah. It's white papers. It's conferences. It'stalking with folks, listening to your podcast,
Ray Wilson. Mhmm. And just seeing what's outthere. Right? And being realistic and testing and just
trying stuff out, experimenting. Right? Drop some modelsinto your lab and see how far you can take
(24:07):
it. Getting up to speed with RAG was quite difficult. Afun summer project for me, testing out some of the different c p
or GPUs rather. Arm, for example, just to help youguys out with getting up and and riding. ARM GPUs are
really tough to customize and get the drivers workingand such. And to why NVIDIA is doing so
(24:30):
well, especially with their stock price, is the the ease ofuse and their flexibility to work with
developers and just somewhat work, the box.
So CUDA is their development driverpackage, and you can configure and tweak how
many GPUs or or resources on the back endthere to toggle with. But it's fun to the tinkering
(24:55):
around with it and trying out these implementationsand checking out stuff, like, on Reddit or on on
Hugging Face, which is turned into the GitHub formachine learning models, is a good place to research
and and find out what's going on. So what Reddit subsubreddits do you recommend? There's a machine learning
(25:17):
and language technology one. Oh, there's also I think there's onefor reg. There's one I was playing around with COBOL
CPP. So it's k with for COBOL, And that wasa fun application to help con with the configuration
in back end. But, yeah, mileage may varywith what type of subreddits they're on. And, yeah,
(25:41):
that's why I'll just keep it that keep it there. Yeah. No. I totally getthat. You just don't know what you're gonna get. Caution.
That's funny. Alright. So when we're talking about AIdriven technologies, I know that you had mentioned a few things that you've been
working on. But maybe just, like, do you have any kind of listof skills that up and
(26:02):
coming folks need to know? Because, technologyis changing so fast. What do you what are you seeing? You're in
startup world, so you see a lot of you see a lot of stuff thatare innovate innovative in general. So what do you what should
we be paying attention to now? Yeah. So inorder to get up and and running, just start playing
(26:24):
around with Python. It's a universal tool forall data scientists. And then most data scientists have been living
in Jupyter Notebooks. So JupyterLab is a goodway to get up and and running. And most
small models and tools and LMs out thereand the integration or orchestration through
(26:46):
a package called, like, Langchain all works with with Python.
Right? So that's the the beautiful thing. The white papers out there, the Reddits ofthe world, on the enterprise side.
Databricks is pretty good. They've got some really good blogs. I use the ragone to get up and and running. Snowflake is pretty good
out there. And then, yeah, just starting to think,of use cases. Right? So with Rag, I
(27:12):
presented at colonel last week, which was reallygreat conference and technically focused. I'll go mini
Defcon. Look at some really good ideas on the red teamersand their pen test reports. Right? Dropping them into to
reg. Now securing them down, locking them down, puttingsome apples on it on who can get access. But at
(27:34):
the end of understanding the the vulnerabilities orwhat the pen tester took in order to exploit a particular
system is, like, your library. Right? Your inventory that you canreference back and forth or get some new creative ideas amongst the
red team by sharing that knowledge. So that was another cool use caseI didn't think of. And then on the GRC side, starting to
(27:57):
see and actually build out myself some of the use cases onjust data privacy laws, regulatory frameworks out there
that are very cumbersome, very long, lengthy,right, PCI DSS, splitting them up, tossing them in
the the vector database, and asking questions. Also,tossing in your own policy documents into the vector database and
(28:20):
then interfacing with it to compare and contrast, like a gapanalysis of Mhmm. Hey. We wanna do business
in Iceland. Here's our policies. Here's Iceland'sdata privacy laws and regulations. What do
we need to do? You'll get a a list of of things that, you know,didn't match on that those gaps And, hey, write me
(28:42):
a new policy that makes me abide by Iceland's. Right? NowI've got a new policy. It's integrated, customized based off of my own
policy. Of course, you gotta go out there and implement it. Right? Can'tjust write it down, but that's another really good use case that that
I've seen. Just gotta make sure that it's not we'renot looking at AI as the easy button because when like, I live in GRC
(29:04):
world. Oh, yeah. We're just gonna we're just gonna use chatcheappt. We're gonnawrite a we're gonna write all of our policies for SOC 2, and we're gonna
be good. And that's the beautiful thing. I was just humans in the loop.
Right? They're gonna need to interpret. They're going to need to bethat last bastion of, hey, is this
bad? And offering up that insightand logistic reasoning that maybe the AI model
(29:31):
doesn't have just yet. So again,a guide and not a particular guard or full stop
fully automated system. Human needs to be in the loopcontinuously. Yep. Completely agree.
Brennan, thanks so much for your time today. Really appreciate you coming on andtalking to us a little bit and making minds explode
(29:54):
like like mine. It's always nice to be, talking to tosomebody that understands what's going on, especially with AI.
So, thank you again. My pleasure. Thanks all. Goodluck out there, everybody. Hope you liked this
episode of DevSec Lead. We'll have other ones coming up in thenext few weeks. Really appreciate you. And
(30:17):
if you could pass along the message and get your friends tosubscribe and we'll keep on learning together. Hope you all have a good one
and talk to you soon. See you.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Bobby Bones Show
Listen to 'The Bobby Bones Show' by downloading the daily full replay.