Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:02):
This is the Generative AI Security podcast brought to you
by the OS Generative AI SecurityProject.
We'll be talking about pertinentindustry issues around
generative AI with our project leads, contributors, and more.
Please click like and subscribe for all of our latest content.
(00:24):
Hey, community, Aubrey here withthe OS Gen.
AI Security Project. And I'm bringing you another
instance of our Generative Security podcast.
So today I have a very special guest.
That's right, I am joined by John Sardaropoulos.
How are you, John? Hey, how you doing Very well and
you? I'm doing quite well.
(00:46):
It's kind of coming up on the end of the day way later for
you. It's like what, 10:00 PM or
something? I Oh no.
It's it's actually about half half, so it's fine.
OK, All right. So that's, that's not terrible.
OK, we'll get you, we'll get youout of here soon, I'm sure.
But we have too much important stuff to talk about, you know,
(01:08):
today. So we'll definitely flow through
that. For those of you that are not
familiar with our project, it is, you know, one of the hottest
projects at OSP right now and that is we've got a number of
different initiatives that are broken out in the Gen.
AI security project. Now.
John, you lead up the agentic security initiative.
(01:32):
Yes, that's. Right, that's correct.
OK, now for anyone who hasn't been to our website,
whichisgenai.osp.org, this is what it looks like and this is
right into our initiatives. So if you're if you're up in the
top menu, you can see initiatives and you'd go to
Agentic App Security brings you right down into here.
(01:54):
So can you tell me and the community a little bit about
this initiative and why, I mean,most importantly, why we created
this? What was the need?
Of course, first of all, I call it the initiative.
My colleague Ron Rosario couldn't join tonight, but he's
been also very active leader of this single step.
(02:15):
So we started this initiative out of the conversations for the
top 10 for LLMS in 2024. There was a genetic, if you
remember back then was a starting to emerge.
We had the excessive agency entry already and we went into
many conversations. I submitted with Emmanuel A, a
(02:38):
entry for vulnerable agents, butwe weren't sure.
And we said instead of creating a new entry or even a new top
ten, let's go away. Let's explore the space, let's
deep dive, let's use a fantasticcommunity to collect data, to
collect understanding and then revisit whether we need the top
ten. And how does this relate to the
(02:59):
top 10 for LLM? Initially we said let's go
through like real developers, let's understand the threats and
mitigations, let's do threat modelling, let's come up with
the the secure guidelines for development, for architecture
for deployment and also the all the ecosystem around security.
(03:20):
And we have created a few documents that have been very,
very popular the we call it affectionately ASIC by the way,
the agentic security initiative.So the we've released the
threatened mitigations, phenomenal adoption about 15
threats in there with this very simple threat model.
(03:41):
And, you know, back then and even now, we wanted to provide
very simple framing of what agentic is, agentic AI and
agentic security is because sometimes people get into
endless conversations, oh, is this and what about this and
what about that? And, and then, you know, we miss
actually the main point, which is it is what it is.
We just need to secure it. So we provided that framework.
(04:04):
We have about 15 threats and playbooks.
We try to avoid producing all the lists because we weren't at
that stage. We just wanted to create a
taxonomy that says, you know, atlarge, these are the threats and
these are the mitigations. 15 threads and 50 page document is
quite big to read. So we've also created, if you go
(04:28):
back, we've created a very graphical navigator that if you
cannot bring that up, that's. There we go.
Yeah, exactly. People can go and download that.
I find that extremely useful in my work and I use threads and
mitigations on a daily basis andArcanus practice.
And so we thought, and with thatwe will actually engage with the
(04:50):
community, not just in the kind of didactic way where, you know,
the community is an audience, where we throw things at them.
But let's take them through the process.
Let's collect feedback. And we realise that you can only
collect feedback in action. If you go to say someone give me
your feedback, they'll stare at you and say what?
So where. So Ali and Ron and others have
(05:11):
started doing hackathons and through that we had fantastic
response. I mean, Ali did the hackathon in
New York with 100 people. She's doing another one in
Defcon. Helen, Helen Oakley created this
new CTF platform that's becominga, in a, by the way, in the, in
the initiative, we have what we call work streams mean
(05:33):
initiatives if you like. And so that's the CTF work
stream where Helen and and Ali have this Simbot, which is a
caption the flag application. So all of that and allowed us to
engage with hundreds and hundreds of thousands of
practitioners and get back feedback.
We've released a document about how do we use threat modelling,
(05:55):
How do we use Maestro as a threat modelling methodology
that Ken Huang has introduced. We also released recently we
worked, you know, we did the open workshop in in RSA that
gave us another set of fantasticfeedback items.
And we released only on Monday the securing agentic apps guide.
(06:18):
And that has become viral. You know, it's the first time I
saw something like within the first day 1000 reactions, people
reading it posting back and say,hey, you know, this is exactly
what we need. So this is really cool stuff and
there's so many people there. Eden beneath Tomer earlier Elias
Karen. I will miss, you know, I will
(06:38):
miss so many people, but that's the beauty of the community that
they've created something that they engage with other people
and. And when they released it, it
became completely viral. On Thursday, we have the last
part of what I call the exploration phase.
In other words, understanding and creating chunky documents.
(07:01):
And that's the state of you already named it.
I can't remember the name. That's that's a shame.
For me, it's the state of AI security and governance.
And that takes a, again, a complementary role.
So securing agenda gaps, look atdesign, architecture, secure
development, encoding, deployment, hardening.
(07:25):
And so now the last part is, well, what's the governance that
you have around that? What's the compliance
requirements that you have, whatthe ecosystem looks like?
And we complete that end to end treatment.
So that what that has done is wehelp the community have concrete
guidelines, but also us now havea solid foundation to step back
(07:46):
and say, hang on a minute, this is a lot of documentation.
How do we now make it more usable?
And this is what I think the top10 comes in and that's where the
excitement starts. So we had many conversations and
we said let's produce Cheat Sheets.
And I know some of the guys in the team are producing some
Cheat Sheets for MCP and A to A.And then we said let's be brave
(08:09):
and and summarize everything into the iconic headline item
that the top 10 is. Well, you said iconic, right?
So we do have a new top 10 coming.
And you did hear that, right? Oh, oh, WASP fans, right, we do.
It's going to be the top 10 for a gentic, right?
(08:30):
So yes, this is where we are today.
And this is an alpha technically, right?
Maybe like a .5 release? Yes, so exactly.
So it's very early days. So unlike all the other top 10s
where people start with expert opinions, we have a head start.
All that feedback that I said, the hackathons, the threats and
mitigations, it's a solid foundation.
(08:51):
So we took that and we translated into entries, 1516
entries. And so if you look, if you look
at the first one over. Memory poisoning.
People will really kind of recognize this is one of the
most well understood. Again, we took what we have in
this certain mitigations and organically evolved it into a
(09:15):
top ten item. The ranking is a little bit
arbitrary. It's just what we have in in the
document, but will evolve that. The purpose with this is to go
through that, define distinctly the scenarios, all the stuff
that we do with the top 10 for LLMS, but also because we have
the top 10 for LLMS, we also want to relay that those two top
(09:38):
10's and that's the value of theagentic top 10 coming out of the
security. The general security project is
that we shouldn't treat them like, you know, there's
something new shiny thing, let'sproduce a top ten.
There is the LLM agentic apps are LLM based applications.
So we went out of our way in thetop ten, in the new top 10 to
(10:00):
relate back to link back to the LLM top ten and make sure that
practitioners and builders and defenders don't just have to
throw away all the security guidelines that we have.
And there is this kind of link and and logical usability there.
Boy it's funny this this one right here too.
(10:21):
LLMO 4 data and model poisoning.This is one of my favourites.
I'm really focused on trying to really come up with some
guidelines and I won't go into it at all, but guidelines for
essentially how to micro segmentyour applications so that you
can kind of avoid some of this stuff too.
But that's just one of those ideas though, that that people
when they join the community can, can maybe contribute.
(10:43):
And I think you guys are hoping for some of that, right?
I mean, I, I did see here we've got 6/16, but you also have this
one down here, that's John Sardaropoulos, Vulnerable
agenda. What is that?
That's that's look, I wanted to,I thought, you know what, let me
be the first contributor. So we want people, this is a
(11:04):
GitHub repository. We want people to contribute.
There is a template. We'll see it in a minute, but I
wanted to give an example of thekind of real life references
that I want to see. We don't want to see academic
sometimes charge DPT, augmented,you know, kind of items.
(11:27):
We want to see things that reflect what people experience
in the real life. And if you can see in this in
this century, I have done a survey of what's happened
recently, tried to connect it. This is the debate we need to
have. It has to be something that
relates to real life experiences.
And right here you I love that you include the the actual news
(11:49):
articles that you know that spawned the need for these.
That's spectacular. Oh, but that's great because
very often people are bombarded with headlines and I wonders
through the great work, always does, to relate all the firehose
of news to the guidelines that we have.
You know, sometimes I go to my customers and I talk about the
(12:10):
AI security and they say, Oh my God, John, I'm drowning and
you're just showing me the water, the headline.
Just give me some practical guidelines.
So that's what we're trying to do.
OK, Now you mentioned the template, so I am going to
actually go ahead and pull this back up here in case, in case
you're wondering how you can actually submit something.
If you if you're looking throughthis and we will include links
(12:33):
in the podcast description. But if you are looking through
this and you see amongst these initial 17 vulnerabilities that
are here, you see something thatyou think is missing.
The last entry here is marked down for the template itself.
And if you want to just clone the repository, go ahead clone
it. And you know, you can make your
(12:54):
own branch. And if you want to do, you know,
a pull request, that's exactly what we're looking for from our
community members. That's exactly.
The process. So, OK, now there there is a
process also by which these things will eventually get
ranked. We're still in the the .5.
But you did show me the yeah, the the community review here.
(13:17):
This is a very early step. We are racing to create what we
call the public draft so you will see more of those votes.
Steve Wilson did a lot of that and we learnt a lot from him on
the top 10 for a limbs. At this stage, all we're asking
is those 1516 threats there and mitigations.
(13:39):
They weren't supposed to be ranked in in the threats and
mitigations. Can you give us the your
preference of how they they rankand then use the there is at the
end a a a, a nice long free textdocument where people can type
anything they like. But guys, if you're going to
propose a new thread, go into what's Obris suggested, which
(14:01):
is, you know, they get don't don't expect me then to make it.
Is the template here is also, you know, someone said, I can't
remember who was Josh or Ryan onfrom Amazon.
I said, well, what about mergingto and we don't have an option
for that. So use that box to say, Hey, you
know, we think those two, they're not really too distinct,
(14:22):
just merge them. But this is just the beginning.
You know, the kind of we want tocreate the first draft and then
go publicly on Monday on the call.
And if you go to that AC page onour website, you will find the
details. Every Monday at 5:30 Pacific
Time, we have a call and we havekind of very, very active and
(14:48):
robust debates about what it should be.
So the Monday that's coming, we are dedicating it to discuss
what should be in the public draft.
So if you hear this in time and you have the time, join us.
We're a very welcoming bunch andwe love feedback.
Awesome, awesome. Now so you mentioned the public
release that was in or the public draft that is.
(15:11):
We talked about it as a .9 when it that's when, September,
October. Yeah, So the the 05 we will kick
off the public consultation August the 6th.
And as you know, we have it's coincides with Black Hat.
So we thought, you know, it's going to rush things a little
bit, but we have so many people from the community in in Black
(15:32):
Hat. Why not grab the opportunity,
bring the the community togetherand get feedback.
So we have this what we call public draft and then August the
6th guys we are kicking off the public consultation and that's
when things will become really at the global scale votes and
(15:52):
you know let's do calls to wherewe review specific
vulnerability, you know the meaty ward.
And then in September we bring the let's call it 09.
Earlier on we said what about 0999 Something before 1?
Yeah, OK, that that works. And there is actually for anyone
(16:13):
who's going to be a Black Hat, I'm going to be there also.
And I'd love to get your feedback on what we're doing
here. So if you see me hanging around
with a camera somewhere and are not afraid to get in front of
it, I'm very happy to to talk toyou a little bit about what your
thoughts might be about this, this particular initiative and
this new top ten that we're building for Ingenic Apps.
(16:34):
Now we've got a couple other things that are coming up as
well that were covered in this LinkedIn post.
Yes. Yes, connects very nicely.
Yes. So that is on that's the August
6th is the global call. That's right, yes.
So. If we look at the global call
and and you can find this on ourLinkedIn.
(16:54):
If you haven't been to our LinkedIn, please go.
We got 13,000 followers in the past couple years, which is nice
and it's the OAS Gen. AI security project again, if
you're not familiar with that LinkedIn group, but one of our
posts here also has has this wonderful, you know, kick off
and the global call here is going to be at looks like noon
(17:20):
to 1:00 Eastern Time. And this will this will be the
big party an amazing graphics here, I got to say.
So that's that. And we also want to make sure
that the community is aware thatthere are localized events too.
So you can see here London is hosted by Kanos.
There will also be Athens, Tel Aviv and Las Vegas.
(17:42):
These are hosted by Tenable it looks looks like.
And and AI and me in Athens, I think that's you know, it's kind
of fantastic that's what's happening because I think it
brings the global community together.
There's there is so much enthusiasm of it's kind of like
I kicked off the London this morning and I've had half the
(18:03):
capacity of Faro people just joining within a day.
I think it's just showcases the global character always has
always been just in black Hadoa hearts be they will be there the
all specific everywhere. And and what we're bringing also
together is I don't know many ofyour viewers know that in the
(18:25):
agentic security initiative, what we have, what we call the
distinguished experts review board.
And what that is, is we believe we are expert backed community
driven organization, which meansthat we have some very well
known names like Apostol Vasilev, the head of adversarial
AI at NIST, Hiram Anderson from Cisco, Chris here, everyone
(18:48):
else, Chris here is on LinkedIn from Aqua.
Oh my goodness, I will forget names.
The chief actors from Oracle. Who else?
Who else? So the Alan Turing Institute
from Microsoft, the the head of red teaming.
So you can you can imagine, you know, the kind of people.
So we will bring all those people together on the call.
(19:10):
All the local events start half an hour earlier.
We'll have pizza and talk and, you know, exchange ideas and
they will all join this massive party on the call and we'll
unveil the the public draft and,and discuss a little bit how
they can participate. So a lot of excitement.
I love it. Fantastic.
Well, I guess that kind of rounds out all the the coming
(19:33):
excitement for the community. You have any kind of final
thoughts before we before we call it a day?
I would say this expert backed, community driven.
In other words, we need the experts.
But the guys, you have the experience of everyday life.
Life isn't just in books and papers, it's lived experience.
(19:54):
So bring it in. Absolutely, definitely join the
project. Join the calls.
Get involved as much as you can,because honestly, if you're like
John and I, you probably understand that this AI thing
needs to be harnessed a little bit.
Exactly. Make it evidence.
Yes, that's right. Thank you, Aubrey.
(20:15):
Well, thank you. I appreciate the time today.
John, I know you're super busy, especially with all this on top
of the day job, so. I appreciate it, Aubrey.
You know what we do at this project, It's just phenomenal.
I love it. So thank you very much for the
time. Me too, and thank you,
community. We appreciate the feedback.
We appreciate you tuning in. If you could do us a favor and
click like and subscribe and go ahead and tell every single
(20:36):
person that you know about this podcast and our project and the
things that we're doing at O Wasp today.
Please do us a favor and help out as much as you can.
Once again, I'm Aubrey King withthe Gen.
AI Security project at O Wasp and you've been listening to the
Gen. AI Security podcast.
Thanks for tuning in. Bye.
(20:58):
Thanks for checking out the Generative AI Security Podcast.
Don't forget to click Like and subscribe and we'll catch you on
the next one.