June 30, 2025 • 38 mins
What happens when your carefully crafted incident response playbook becomes worthless? Cody Sullivan from OpsBook reveals the brutal truth about tabletop exercises: most organizations are practicing with medieval armor for a drone war. From 70-participant, 6-hour exercises spanning three continents to the harsh reality of insider threats, this conversation exposes the gaps that could leave your organization bleeding when the real attack comes.
Key Topics Covered:
- Why "tribal knowledge" is your organization's biggest security risk
- The insider threat scenario that makes every tabletop exercise go sideways
- How AI is revolutionizing incident response preparation through OpsBook's ontology
- Why your playbooks are useless if hackers have them too
- The "Derek Jeter approach" to cybersecurity preparedness
- From real estate to tech: spotting warning signs before the industry shift
The crew shares fresh insights from a recent school district tabletop that exposed critical single points of failure, while Cody demonstrates how modern organizations are turning decision-making into muscle memory, not just memos. This isn't theory—it's the frontlines of organizational resilience where one overlooked vulnerability could trigger catastrophic failure.
Like, share, and subscribe for more in-depth security discussions that prepare you for tomorrow's threats, not just today's compliance checkboxes!
#tabletopexercise #incidentresponse #cybersecurity #infosec #AI #opsbook
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:04):
Welcome to the Audit presented by IT Audit Labs.
My name is Joshua Schmidt, yourco-host and producer.
We're joined by the usualsuspects Eric Brown, our
managing director, and NickMellum.
How are you guys doing today?
Speaker 2 (00:15):
Doing well, thanks Excellent, ready to jump in.
Speaker 1 (00:18):
Awesome.
Yeah.
Well, our guest today is CodySullivan from OpsBook.
I spent a little time with Codyon some pre-production to talk
about the project he's beenworking on, but we also wanted
to have a discussion aroundtabletop exercises and Eric's
experience working with them,and Nick is our social
engineering guru.
So we'll have a lot to talkabout today, but first I wanted
(00:38):
to get a little background onyou, cody, and what you've been
working on recently.
Speaker 3 (00:41):
Yeah, I'm working on a lot.
I mean, I think you probablyhear this a thousand times from
any founder that's founder-ledsales and rocking and rolling
with a small team.
We're just trying to bring onnew customers and build a lot of
product value behind it.
We've recently launched two newproducts, which is exciting.
One's in beta right now and theother is fully developed.
(01:02):
But trying to figure out themessaging and all the fun stuff
on the sales and marketing sideis always a challenge.
Speaker 1 (01:07):
Cool.
Well, one thing I learned aboutyou and then I'll kick it off
to Eric and Nick but you hadshared with me that you left a
successful real estate careerbecause you saw the writing on
the wall that tech was going totake over.
What warning signs did you spotand what were others missing at
that time?
Speaker 3 (01:25):
Yeah, it's actually really funny.
This is going to sound like Ihad a little bit of a crystal
ball, but this is humble as muchas it is.
This is a true story.
The first day that I got myreal estate license, I got
invited to a big globalconference where it's one of the
rah-rah sessions.
Get 20,000, 30,000 agents readyto go.
(01:47):
I'm not going to name the brandthat I hung my license under,
but we went to this bigconference and, mind you, I have
not sold a house yet.
I have knocked on maybe threedoors and am just cutting my
teeth.
And I'm sitting here listeningto the owner of this real estate
brokerage talk about protectingdata from Zillow and Redfin and
(02:10):
some of the online applicationsthat a lot of consumers back in
2017 were using primarily andstill do today.
And he's going on this rah-rahchant about how we need to
protect our data and it's theagent's data that's important
and we're the ones that arespending the money to collect
these leads and build thesebusinesses around Zillow
applications and everything inbetween.
(02:31):
And then he turns around and hesays so I protected your data
by building our own internalapplication to be able to
collect all this resources andinformation to be able to sell
to your clients and thenimmediately to answer the
question.
It dawned on me in that moment.
I looked to the guy next to mehaven't sold a house yet and I
go.
He's doing the exact same thingthat he's telling us we
shouldn't be doing.
(02:51):
We don't own that application.
And that's really when itdawned on me.
Even after several years, afterbeing a real estate agent and
finding a successful career, Ialways knew that I wanted to get
into technology because I knewthat's where the gold was, was
in the data and being able toreally provide products and
solutions at scale.
And that was really when thelight bulb clicked for me.
(03:13):
It took me a few years becausethe money was good and I enjoyed
it, but I got out as fast as Icould.
Speaker 1 (03:18):
Well, I'll pass it over to Eric.
On that note, Eric, you justcompleted.
Can we talk about the tabletop?
You just completed?
Speaker 4 (03:24):
Yeah, we did a tabletop for a school district
recently and it had some prettygood findings of the respective
teams needed to do to cometogether in the event of an
actual cyber incident, how theywould work together and some of
(03:50):
the things that maybe weren'treally apparent, where you had
some single points within thecommunication chain.
You know, maybe they weren'treally realized until it's like,
wow, this person is doing allof these things.
In the event of a real worldscenario where you're going to
be on multiple bridges, yourphone's going to be banging off
(04:13):
the hook, you're going to begetting text messages, people
are going to be showing up inyour queue, the planning goes
out the window when the shotsget fired right or some
derivative therein.
But the tabletop really bringsthat to light and it does allow
organizations to drill and trainand then if there's an area
where they need to get a littlebit more focused or specific,
(04:34):
they can, and then they can gowork on plans.
They can bring in people ifthey need to, to really iron out
some of those areas that mayfall down in the event of a
real-life scenario and hopefullythey never have a real-life
scenario, but I'm sure as you'vedelivered these, you see that
aha moment and it's kind of funto be there with an organization
(04:57):
as they're going through thatexperience.
Speaker 3 (04:59):
Yeah, I mean, I couldn't agree more.
We've never.
Whether it's a practitionerthat uses our product to
facilitate these as a thirdparty VC so our MSP or whether
it's a direct organization thatis using our solution to scale
or augment you know theircurrent tabletop program.
We've never heard anybody, toyour point, walk out of one of
these engagements and go, wow,that sucked and I didn't find
(05:21):
any value out of it, right?
But I think you really nailedit and you know.
I think you might have said thatNick had a comment about this,
but you know, a plan is only asgood as you can implement it.
And you know what we reallypride ourselves in is because
most of these organizations,whether it's a school or a
manufacturing firm or even atechnology company, most aren't
(05:43):
even testing these plans.
They have a plan in place butthey're not actually testing
them.
And if they are, it's so rarethat our job is to come in and
break these plans, like, at theend of the day, we want to break
them so they can rebuild themand bring it back stronger.
Speaker 2 (05:58):
The culture that we've been seeing with a lot of
our clients is they maybe knowwhat a tabletop exercise is but
they've never participated andthey might have a lot of
branches underneath theirorganization that would funnel
in or have you know to do workif something happened.
Real life happened and I thinkthe quote that Eric was talking
about is something I've saidmany times is when I was in the
military I had a leader tell methe more we sweat in peace, the
(06:21):
less we bleed in war.
So this is directly.
You know I mean differentsituations, right, but if we
plug that into a tabletopexercise, we can show people how
important it is that.
You know, maybe something'sgoing to be pretty clunky, no
matter what right People are.
You're going to be in somescramble effect, no matter what
if it's real life.
But we can take the stress outof it here, slow it down, have
(06:45):
that teaching moment so we canbetter prepare them when they're
actually in.
You know, lack of better termconflict.
Speaker 1 (06:52):
Yeah that's exactly right.
Speaker 3 (06:53):
You definitely want to exercise and practice these
engagements before you get intothem.
You sound like a baseball fan.
You know it'd be like meplaying college baseball and
never taking batting practice upuntil game day.
Speaker 2 (07:03):
You know, it's just no way you're making contact.
Speaker 3 (07:06):
Yeah, it seems odd in that, in that context, but
that's how a lot oforganizations are running.
You know they have, they've gotthese plans on paper and you
know to your point when, when,when the proverbial stuff hits
the fan, you know they'rebleeding, oftentimes because
they've never actually taken aswing or the best they can do
right now is they've got a calltree, right People, they're
(07:27):
going to start calling, butnobody has action items.
Speaker 2 (07:31):
Who's doing what right?
We should be on the same sheetof music before something even
happens, You're nailing it.
Speaker 1 (07:34):
It sounds like by Eric's description.
It kind of gets into that highschool project zone where
there's one person kind of doingmost of the work and everyone
else is kind of waiting for acue.
What was the aha moment for you, cody, that made you realize
this was an issue that you knowyou could help identify and
maybe help people solve?
Speaker 3 (07:53):
At my co-founderized previous firm, we were working
with audit and regulatorycompliance software, so we were
seeing a lot of SOC 2 prep, alot of CMMC and FedRAMP and as
we started to discover moreconversations with enterprise
and men, enterpriseorganizations, predominantly
CISOs and the B-CISOs that weresupporting those efforts, we
were having a lot of discussionaround how do I actually know
(08:15):
that my organization and thepeople that are in charge of
these playbooks and, mind you,these are enterprise
organizations that arerepetitiously testing these
tabletops they do know what theyare.
They are running these on areoccurring basis, as much as
they can.
But they started to ask thequestion how do I measure this
stuff?
First and foremost, and that wasa big aha moment for us.
(08:37):
At first we thought we can buildsome KPIs and some measurable
benchmarking for enterpriseCISOs and VCISOs to be able to
say, rather than just goingthrough a communication exercise
in a tabletop, how do I measurethe projected outcomes that
this might impact and what arethe actions that one might take
(08:57):
place?
And to answer your questionspecifically, our experience in
audit and regulatory compliance,we had a little bit of a knack
for thinking about how controlsaligned to audit evidence,
evidence preparation.
We looked at the samerelationship between the actions
that an individual third party,supplier or a system might take
(09:18):
within an organization in thesame breath is in the event of
an incident, as a control alignsto an audit for preparation, we
want the actions to align tothe coverage for an organization
in the event of an incident.
And that was our aha moment andyou know a few discussions with
some some really well-respectedCISOs and information security
(09:40):
leaders at various organizationsin technology and outside.
We just said you know, this isa problem that we can bring
automation to and reallystreamline for these guys, that
they've never had before andit's just been gangbusters.
It's been fun.
Speaker 4 (09:54):
Cody, what are you doing with AI in your platform?
Are you using it at all?
Have you brought it in forrole-playing scenarios or kind
of?
Where are you on that journey?
Speaker 3 (10:06):
yeah, um, this is a great opportunity to to maybe
give a little sneak peek on ouron our newest product that we're
calling pulse right now.
Um, to answer your question,we've always had generative ai
as an element to generate someof the textual script and story
lines that that typically comewith the creation and
facilitation of these exercises.
(10:26):
That's been the bread andbutter with our legacy products
since the beginning.
I think that one would findthat the common individual
that's organizing tabletopexercises, designing and
facilitating these, the commonpain point that they're having,
is that creativity.
On the front end, I can goonline to an ISACA template, buy
(10:47):
a ransomware scenario and thenswap out the name of the company
and call it a day.
Or I can use generative AIthrough a solution like Opsbook,
which, as far as we're aware of, is the only solution that's
doing this throughout thetotality, where I can have a
subjective scenario that'screated in minutes that is
entirely relevant to the contextof my organization.
(11:10):
So we've definitely usedgenerative AI throughout all of
it, but we've also built anontology on top of that to make
it super, super, exerciseresilience focused.
We want to be able to create anontology on top of that
generative AI to say, hey, ifyou are a manufacturing firm, a
supply chain distributionfacility, if you're an energy
(11:32):
company that needs to supplyenergy to individuals and
consumers, how can we creategenerative AI driven tabletop
exercises but have an ontologythat makes it primarily focused
for our specific industry needsand our specific metrics that we
need to measure, to executethat coverage that we need to
execute?
So, to answer your question,generative AI is entirely
(11:55):
throughout the totality of theproduct.
Speaker 1 (11:57):
I like that vocab word, ontology.
I had to look that up reallyquick.
That's a good one.
Speaker 3 (12:03):
Well, I didn't know until 30 seconds ago, just
disclaimer.
Speaker 1 (12:07):
I had to look it up so that one hasn't come up in my
research, but I was hoping wecould get into some stories.
I know, eric, I'll give you achance to think about it, but I
asked this question to Cody.
But I'd love to hear you knowabout some of the actual
tabletop exercises you've doneand some of the findings maybe
that have come out of it.
You shared with me that yourecently did one with 70
(12:28):
participants and 40 injects.
Was it what chaos or clarityemerged from that process, or is
there anything that surprisedyou or might be fun to share?
Speaker 3 (12:38):
Yeah, from that particular engagement.
I think the exciting thing forus as an organization was seeing
how many people were involvedin that.
We had three differentcontinents that were included in
that and so it wasgeographically dispersed.
So that was exciting for us tosee, historically, what's always
(12:58):
been a face-to-face interactionor a smaller Zoom engagement.
With the technology that we putin place in OpsBook we've
actually been able to scale that.
So that was just kind of thefirst aha moment of like wow,
this is really applicable forlarger engagements.
We now give intercontinentalorganizations accessibility to
tabletops like they've never hadbefore, because in a Zoom call
(13:20):
you can't collect theinformation, you can't make it
subjective, you can't go througha PowerPoint deck and also see
the level of engagement thatwe've been able to create within
the conduction portion of ourproduct.
So that was a really excitingmoment for us in that particular
tabletop.
But, to be honest with you, thismight not be the answer that
you want to hear, but we reallysolve for a lot of the mundane
(13:43):
stuff that enterpriseorganizations, you know, don't
want to necessarily have tomanually go through.
So in an organization like thatthat's large, they're running
so many exercises that areoutside of the realm of just
cybersecurity.
It falls into IT process.
It falls into supply chaindistribution.
You know pathways.
(14:04):
It falls into emergencyresponse plans for a natural
disaster or an earthquake.
Work with financialinstitutions that run exercises
about.
You know, unfortunately, crisisand shooter scenarios, you know
.
You mentioned, obviously, ericrunning one with a school.
You know these are real worldscenarios that we just get
really excited about the optionto take not crazy scenarios, but
(14:29):
the scenarios that need to berun, that are actually going to
happen in the real world, andbring those to scale.
Speaker 2 (14:35):
Cody during that big 40 inject tabletop.
You guys did.
How long does that tabletoplast?
Speaker 3 (14:43):
Yeah, we chuckle because that particular one was
about six hours and I don't knowa person in this world that
would want to be in a tabletopfor six hours.
But uh, we built some coolfunctionality to where you can
break these things up and so, uh, you know those, those 40
injects to your point.
You know, think of those as uhkind of a chronological or a
(15:03):
sequential timeline in the eventsimulated event.
So you're not always testingeveryone at once in those long
events, so you might have fouror five injects that go against
one particular department, fouror five that go to the next.
You can kind of put some peopleon ice and let them go take a
break and do some things.
But it went quite smoothly,guided by the product you took
(15:25):
started answering my nextquestion.
Speaker 2 (15:26):
I was going to ask if you guys are breaking that up
by maybe by department or groupof people, letting some, like
you said, take a break.
But yeah, that's awesome thatyou guys can do that with the
tool.
Speaker 3 (15:37):
Yeah, one of the biggest things that we've found.
And I don't know how manytabletops you guys have done,
but in almost all of them wealways find that there tends to
be a rabbit hole.
Right, there's either somebodythat just got hired on as a part
of their onboarding theyhaven't gone through their
training, they're a little bitinfant in their role or somebody
super experienced and says I'vebeen here for 20 years, I got
(15:58):
all the tribal language down.
That's not how we do thingshere, and in both cases there
tends to be this rabbit hole ofdialogue.
And so, to your point, wecreated this concept of what we
call a branch, which is think ofit just like a breakout room in
the product, and so we've givenpoints for those discussions to
be had, the notes to be loggedand then also to be brought back
(16:19):
to the guide rails of what theexercise was meant to be.
Had the notes to be logged andthen also to be brought back to
the guide rails of what theexercise was meant to be.
So we've thought of a fewlittle fun things like that in
the product.
Speaker 2 (16:28):
I just picked up on one thing you said.
You said tribal knowledge.
That is something we, I think,are always going to see, but
specifically we were doing notnecessarily a tabletop exercise,
but we were doing some planningbecause somebody was going to
retire at a client we wereworking with.
This is like a year or two agonow and he had been there for it
was like 30 years and he was aleader of the IT space there or
(16:50):
the IT group, and it was alltribal knowledge.
There was virtually nodocumentation and that was a lot
of our conversation.
Was you know, catastrophicevent?
Whatever it is, you know?
This is we're getting to thatpoint.
Now if he's leaving, what areyou guys going to do if he's
gone tomorrow?
So that planning getting readyfor a tabletop exercise for a
(17:11):
lot of organizations just can bea good exercise before you even
get there, because they're youknow.
When you're telling them, ok,we're doing a tabletop in a week
or two, whatever the timeframeis, they're subconsciously at
least getting ready right orthey're preparing.
So that can be helpful.
Speaker 3 (17:26):
Yeah, you're nailing it.
I mean, at the end of the day,that travel knowledge from that
individual that's been there for30 years.
A lot of our customers areusing us to replicate that
throughout the playbooks thatthey need to continue after that
person retires.
So the backend element toOpsBook is not only the creation
of facilitation, but a hugevalue add is what we like to
(17:47):
call the after action reviewreporting template, and one of
the parts of that is that thiscan act as a source of truth or
a repository for people thatdon't know what am I going to do
in the event of this scenarioof this guy that's been here for
30 years that I've never youknow, I've never done this
before.
They can refer back to thoseafter action reviews, quickly
(18:08):
filter through them and seeexactly what he would have done
in that instance.
Speaker 2 (18:12):
So this is after the tabletop's been done.
Is there a timeframe that theycan go back to this information
or are they extracting it out ofops book and it's a takeaway
where they can use this ashomework or, you know, every
year, every six months, whateverit is, they can go back and
review, make sure their house isin order yeah, it's all the
above.
Speaker 3 (18:30):
To be honest with you , nick, some of our customers
use it as a source of truth tojust refer back to for
continuous training.
Or even just look at theiterations of playbook changes
over time Like, hey, a year agowe found these gaps, we've
shored them up.
Now we've tested them again.
Now let's look at the newplaybook.
We've had organizations use itfor training perspective, where
(18:50):
they take an app lecture reviewand then they recreate a
scenario to use it for trainingand onboarding to be able to
effectively test people in thoseroles.
They can also use itinstantaneously after the
exercise for audit andregulatory compliance purposes.
We come from that background,right.
So we have that in mind to say,hey, if you're going to be
going for a CMMC or a FedRAMP oreven an ISO 27001 that requires
(19:15):
these things, how can we push areally, really specific example
of this evidentiary item orrequirement into a GRC product?
And so we've got, we've donethat as well.
Speaker 2 (19:25):
Yeah, I think you're getting at.
The same thing I've beenthinking about is, you know, for
doing a tabletop exercise today, we don't want it to just be a
two to six hour event.
You know this should be anongoing right.
Once you're done here, itshouldn't just be a check in the
box hey, we're done, we did ourtabletop.
Maybe it's for compliance, ormaybe it's their cyber insurance
says they need to do a tabletopexercise.
There could be many differentreasons they're doing it, but
(19:47):
you know we really want it to beokay.
You learned this today.
Now let's apply it and keepreviewing to make sure we get
that into our tribal knowledgeor our standards.
Speaker 3 (19:57):
Yeah, we, we like to.
We like to coin a phrase onthat.
You know we help teams turndecision-making into muscle, not
a memo, and that's what we liketo say about the reoccurring
idea of tabletops beingsomething that's accessible and
reoccurring.
Speaker 1 (20:13):
That's awesome.
So not to over-egg the puddinghere, but if someone's you know
looking to maybe work for ITAutolabs or thinking about, you
know, opsbook or implementingsome software like this, I like
to kind of give them a highlevel overview and maybe some
info.
Information from Eric as a CISO.
You know your experiencerunning these things and you
(20:34):
know I'm always looking for thejuicy stories right or for
outlying kind of events.
But maybe you could shed somelight on those types of things
or how you prepare or get peopleto do their homework once the
tabletop is concluded.
Speaker 4 (20:47):
Yeah, sure.
So I think it's a couple ofthings on the tabletop side and
all of the real world scenariosthat I've been involved with and
have triaged, either as anincident person responsible for
the incident, the actualincident response or, you know,
(21:07):
as the CISO.
They've never actually marriedto a tabletop experience, in
that the real world scenario isgoing to be pretty mundane At
least the ones that I've beeninvolved in typically comes in
(21:28):
over fishing or you get an alertthat something doesn't look
right and there's activity thatis not consistent with usual
behavior, and then you go andyou find out that yes, you know
we've had a breach, we've had anincident in the environment and
now we have to recover fromthat incident.
(21:51):
And you know you go intocontainment and you're going
through all of the steps thatyou would in a tabletop around a
ratification and the aftermaththere of recovery and where it
differs in the real world is theamount of distractions and
(22:14):
meetings and just follow up anaftermath that you're pulled
into in the course of resolvingwhatever the issue is.
Because there's, if you'reworking in an organization
that's got regulated data,you're trying to figure out well
how much data is potentiallyexposed and all of the different
(22:35):
third parties that you'reworking with to resolve the
incident and you can, in a safespace not in the heat of the
moment talk about well, okay,yeah, we can absolutely do a
recovery.
What happens if we recover andthe infection is still in place
and the threat actors are ableto turn that up?
That's easy to do and welcomedin a tabletop exercise where you
(23:00):
can then go down differentderivatives of okay, yep, we're
going to restore it.
And then, if it's still there,what do we do?
Or no, we're not going torestore, we're going to rebuild.
You know whatever thosescenarios are.
But in the aftermath, when youhave a, say, a billion dollar
organization that has livecustomers that either can't get
(23:21):
ahold of somebody or they'rebanging away on the phones, you
have news reporters, you haveall of this commotion happening
and you have the leadership ofthe organization saying you need
to recover, that we have to beup.
There's swearing, there's fistspounding on the table.
We got to get that back up.
We got to get it back up rightnow.
(23:42):
And if you go through thatexercise and you drill that over
and over again and you havethat conversation with those
leaders, like, yeah, you'regoing to be hot under the collar
.
The board's going to be callingyou.
You're going to have to makethat decision.
You're going to have to pushback when I tell you, no, we
cannot restore.
You're going to have to trustme, and here's why.
But in the heat of the moment,if I'm telling you, you have to
(24:04):
trust me, and here's why you maynot hear that unless you've
seen all of the steps that theIT organization has already gone
through, and why you can trustme when I'm telling you that.
So that's why I really love thetabletop exercises, because you
can hit pause on any one of thespecific scenarios and you can
dive in and you can reallyreplicate what that's going to
(24:27):
look like in a real environment.
Speaker 3 (24:29):
When we kick things off.
We originally identified as anautomated tabletop solution for
that exact reason you know someof the generative AI, you know.
Just to iterate and echo whatyou're saying, we've recently
adapted more of anidentification of exercise
resilience, because the wholepoint of these tabletop
exercises is a proactive and apracticed and a learned behavior
(24:54):
leading up to the event.
But you know you nailed it Likethe real world scenario on the
response side is what's mostimportant is can we accessibly
access that coverage and do weknow how to execute it in the
event of a real world, uh,incident?
And you know I mentioned kindof teasing our newest product.
You know that we're callingpulse and I mentioned an action
(25:15):
catalog and that's exactly whatwe're trying to do, eric, is
we're trying to.
We're trying to associate theactions that one might have
practiced in a tabletop exerciseand make those readily
available in the event of anincident.
So if we can categorically takepracticed, simulated behaviors,
responses, actions, systems,suppliers, and almost build in
(25:41):
conjunction with that ontologylike a master spreadsheet of
approved actions that anorganization might take, and I'm
talking down to the granularlevel grab the wheelbarrow,
throw the servers on there andgo out in the earthquake.
If we can categorically havethose things organized, then
when it does come time torespond, you don't have to think
(26:02):
twice, even though you havepracticed it.
You've got those readilyavailable at your fingertips and
that's what we're really goingwith our newest product, Ups.
Speaker 4 (26:08):
But Pulse and I think that tabletop exercise and
maybe that's what you're talkingabout in your new product set
where people can simulate thosethings, where they can go and
say, ok, you know, this is whatis happening.
Here is the scenario.
Let's just iterate on.
What are all of the things thatwe're going to face?
(26:29):
What are they going to tell usthat we can do or you know we
can't do?
Let's bring in some of thosethird party, realistic, real
world scenarios where we have aserious Microsoft issue.
We're going to call Microsoft,we're going to open a SEV1.
Microsoft's not going to jumpon the phone right then and
solve our issue right.
It's going to get passed around, it's going to get escalated.
(26:51):
They're going to want to knowif we have 24-7 support and all
of this other nonsense beforethey even engage with us on this
SEV1.
So we have to really bethinking of, in the heat of the
moment, how is a scenario wherewe need third party help?
Can we even get it?
And if we can't, what are someof the things that we could be
(27:13):
thinking about doing rather thanjust waiting and saying oh, we
have a SEV1 with Microsoft, youknow they're going to come in
and solve all of our problemswhen, in practice, we know that
that's far from the case Exactly, and that's really why we're
going to come in and solve allof our problems when, in
practice, we know that that'sfar from the case Exactly, and
that's really why we're tryingto spin tabletops on his head a
little bit with our newestproduct.
Speaker 3 (27:29):
So imagine having a playbook that's prescribed in
place of approved actions thatyou, as a CISO, allows your
organization or third parties onyour behalf to execute in the
event of an incident.
We've got softwares in placefor these things that can
automatically make decisions andcreate tickets and execute
accordingly.
We've got individuals.
You know there's organizationalroles and responsibilities that
(27:51):
fall into that.
It's a fairly dynamic process,but if you can categorically
have those identified, ratherthan starting with the tabletop
and then moving to approvedactions, we're flipping it on
its head with the newest product, which is let's start with how
you'd actually solve the problem.
Let's start with the movingtruck and throwing the servers
in there, and then we canmitigate the gaps after.
(28:12):
And by doing that we are nowusing, with the help of AI built
in the product and the ontologythat makes it subjective, we
are now able to take a scenarioand test it hundreds of times
with the software in betweenthese engagements.
So when you show up to atabletop exercise that might be
(28:32):
testing a business continuityplan annually or an emergency
response plan or an incidentresponse plan.
Every 90 days you're showing upwith the most updated version
of that action catalog ofsolutions that you might take on
behalf of your organization andif there are gaps or
fine-tuning that you need to doas a mid-enterprise or an
enterprise org, you canbasically say that to a degree
(28:56):
of certainty.
We've tested this a hundredsimulated times and now we
brought it to the finish line onexactly how we want to perform
on it.
So you are now far moreprepared than just running this
once every 90 days or once every12 months.
Speaker 1 (29:09):
I can really relate to that.
Growing up in a remote part ofMinnesota, the North Shore kind
of had to be thinking on ourfeet and finding ways to solve
problems with what we had theresources available to us in
that moment because we couldn'tjust order something on Amazon
and is available to us in thatmoment because we couldn't just
order something on Amazon and Iwas two hours away from a bigger
city.
So I think that is kind oftranslated into my professional
(29:30):
life.
But also I think that's how weview things at IT Audit Labs too
.
It's just like an overallculture of getting things done
and it sounds like that appliesto Opsbook as well and you can
kind of bake some of that stuffinto the inputs, am I correct?
Speaker 3 (29:47):
That's absolutely right.
Yeah, and the beauty of it isthat if we're partnering with an
IT audit labs or a third partyconsultant or a VC, so we've
really designed the system tokeep that stuff in mind.
So whether it's an organizationor a consultant in a smaller
organization, it's really comingin to help.
You know, those inputs areimportant for the subject matter
(30:12):
experts, but if we can testthem at scale, even for smaller
organizations, then we're nowdecreasing response times.
You know, we're strengtheningteam cohesion at smaller
organizations with small teams.
We're strengthening teamcohesion at smaller
organizations with small teamsand we're streamlining the
relationship for third partieslike IT Audit Labs or
consultants on the side toeffectively come in and say I
can now make your smallerorganization operate like an
(30:35):
enterprise organization with atool like this in place.
Speaker 1 (30:42):
I know we work a lot with the sled in that space,
sled space and I just wascurious if Eric had kind of a
go-to like monkey wrench youmight throw into a tabletop
exercise to kind of get thepeople to think on their feet or
create a culture of problemsolving.
And then maybe we could getCody to even follow that up for
how the ops book might approachsomething like that.
As we talk about macro andmicro, is there kind of a go-to
(31:03):
monkey wrench you like to throwinto a pen test or, sorry, into
a tabletop exercise that kind ofgets people to think on their
feet?
Speaker 4 (31:10):
We like to engage with the person who is
sponsoring the exercise andreally understand what it is
that they want to get out ofthat exercise.
Right, is it more on thetechnology side?
Is it more on the leadershipside?
Is it more on the leadershipside?
Is it more just on thevisibility side?
So certainly we'll come up witha scenario based on what that
(31:32):
person is looking for.
But the question that we alwayslike to ask maybe it's the the
monkey wrench is who gets topush the button Right and at the
end of the day, when the when,when the scenario is going on or
the real life scenario ishappening and there's something
(31:53):
that will be impactful to thebusiness, where you're maybe
taking some some very criticalsystems offline to resolve an
issue, or you know you have tomake that call to do something
that is largely impactful, whois the one person that can make
(32:16):
that decision?
And you know, sometimes we hearit's well, it's like a group of
people and we really try todrive down, like who is going to
push that button when the timecomes?
And that's hard fororganizations to answer a lot of
the time.
Speaker 3 (32:34):
The one that seems to be tested the most is obviously
ransomware, but what they don'tinclude in that typically is
insider threat.
They don't include in thattypically is insider threat.
I mean, it's just, it's one ofthe ones that every time we see
an inject in a tabletop exercisethrown in with an insider
threat, nobody seems to learnhow to respond or know how to
respond, and it's almost likethere's a cultural offense to.
(32:58):
Even in a hypotheticalsimulated scenario in ops book,
how could I possibly imaginethat John who's in the cubicle
next to me could possibly be thereason that we're being
attacked right now?
That's always a fun one tothrow at people.
The responses seem to go youknow just every which way under
Sunday, but it also is commonlyidentified as a gap and a great
(33:21):
thing to test whether it's aransomware scenario or just an
insider threat by itself.
The second one that's alwaysbeen fun and I think this is
relatively common knowledge inall of cybersecurity and even
physical security is if you'regetting attacked by a threat
vector or a hacker.
For example, they've had accessto your system far longer than
(33:42):
yesterday, for example, they'vehad access to your system far
longer than yesterday, and whenwe see curveballs or injects
that are thrown in thosescenarios around, what happens
when you have updated playbooks,but so does the hacker?
They have access to yourplaybooks.
How do you respond when yourplaybooks are now null and void
in totality?
(34:02):
You cannot use any of them, andso when we throw that inject, I
think it dawns on a lot ofmature organizations that even
are testing these exercises on areoccurring basis.
They go oh crap, we don't havea plan B for these scenarios.
And if we do follow the scriptof these playbooks or these run
books in the event of anincident, he knows my moves
(34:24):
minutes, hours before I take itright.
So I think having somecontingency plans in place is
always a fun one for us todiscover in some of these
exercises.
Speaker 4 (34:32):
I like that you brought that up, cody, because
it really articulates howorganizations should be treating
and drilling the playbookswhere it's not really a script,
and I think we saw that withsome of the more popular tv
shows um in in the late 90s,like I think it was maybe
friends and and seinfeld.
(34:55):
I think those are the twoexamples where they weren't
necessarily scripted, where thethe scene was written and the
actors were given the generaldescription of the scene, but
the actual words that they usedwere open for the actors to
interpret, to convey the messageof the scene, versus reading
(35:18):
and memorizing verbatim.
Here is the exact script.
I'm going to say this.
This person is going to saythat verbatim.
Here is the exact script.
I'm going to say this.
This person's going to say thatBecause in a real world
scenario you can never drillexactly what's going to happen.
You mentioned the insiderthreat.
There could be an insiderthreat piece.
There could be a person who iscritical to the success chain,
is sick or out on vacation orwhatever that is.
(35:40):
But if the organization isdrilling the actual context and
not exactly the steps in theplaybook, I think they'll be
more successful, and it soundslike that's what OpsBook can
help them do.
Speaker 3 (35:53):
Yeah, I'll make two quick comments on that.
I'm a baseball guy, love thecaptain.
I talked to a guy that's a BoSox fan today but I love Derek
Jeter and I'm not a diehard NewYork guy but I love the cap and
in his retirement he really saidit best for a baseball analogy.
That I think aligns really wellwith cybersecurity and some of
these resilience plans.
(36:14):
As he said, I've simulatedevery potential play that might
present itself to me before thepitch is ever thrown potential
play that might present itselfto me before the pitch is ever
thrown and I think we really tryand take that mentality with
the action catalog and our newproduct to be able to say why
should tabletops be a staticengagement?
(36:35):
Why should they not beconstantly simulated in real
time for enterpriseorganizations?
So in the event that an injectdoes happen, where somebody has
access to my most updatedplaybook that I'm housing in my
GRC or housing in a third-partysystem, I can log into OpsBook
and immediately have the mostiterative updated version of the
solutions, the steps and theactions that I would take place
(36:56):
in the event of this breach.
Because my process, my people,my onboarding, my ticketing
system everything's changed inthe last 14 days anyways.
So why would I be using a 14day old playbook when I can use
a two hour old one?
Why go to battle with medievalarmor and a long sword when
you're playing a war that'sfought with drones?
You know what I mean.
(37:16):
You really got to step up tothe modern day when it comes to
resilience, and that's whatwe're trying to do with our
newest product, for sure.
Speaker 1 (37:24):
Well, thank you so much for your time today, cody.
We've been talking to CodySullivan from OpsBook.
My name is Joshua Schmidt,co-host and producer.
I've been joined by Eric Brownand Nick Mellon from IT Audit
Labs and have a great day.
We publish episodes every otherMonday.
We're on Spotify, youtube,amazon, apple you name it.
So wherever you find yourpodcast, give us a subscribe, a
follow and a comment and areview.
Speaker 4 (37:48):
If you have the time.
Thanks a lot for listening.
You have been listening to theaudit presented by IT Audit Labs
.
We are experts at assessingrisk and compliance, while
providing administrative andtechnical controls to improve
our clients' data security.
Our threat assessments find thesoft spots before the bad guys
do, identifying likelihood andimpact or all.
Our security controlassessments rank the level of
maturity relative to the size ofyour organization, thanks to
(38:12):
our devoted listeners andfollowers, as well as our
producer, joshua J Schmidt, andour audio video editor, cameron
Hill.
Cameron Hill, you can stay upto date on the latest
cybersecurity topics by givingus a like and a follow on our
socials and subscribing to thispodcast on Apple, spotify or
wherever you source yoursecurity content.
Welcome to the Audit presented by IT Audit Labs.
My name is Joshua Schmidt, yourco-host and producer.
We're joined by the usualsuspects Eric Brown, our
managing director, and NickMellum.
How are you guys doing today?
Speaker 2 (00:15):
Doing well, thanks Excellent, ready to jump in.
Speaker 1 (00:18):
Awesome.
Yeah.
Well, our guest today is CodySullivan from OpsBook.
I spent a little time with Codyon some pre-production to talk
about the project he's beenworking on, but we also wanted
to have a discussion aroundtabletop exercises and Eric's
experience working with them,and Nick is our social
engineering guru.
So we'll have a lot to talkabout today, but first I wanted
(00:38):
to get a little background onyou, cody, and what you've been
working on recently.
Speaker 3 (00:41):
Yeah, I'm working on a lot.
I mean, I think you probablyhear this a thousand times from
any founder that's founder-ledsales and rocking and rolling
with a small team.
We're just trying to bring onnew customers and build a lot of
product value behind it.
We've recently launched two newproducts, which is exciting.
One's in beta right now and theother is fully developed.
(01:02):
But trying to figure out themessaging and all the fun stuff
on the sales and marketing sideis always a challenge.
Speaker 1 (01:07):
Cool.
Well, one thing I learned aboutyou and then I'll kick it off
to Eric and Nick but you hadshared with me that you left a
successful real estate careerbecause you saw the writing on
the wall that tech was going totake over.
What warning signs did you spotand what were others missing at
that time?
Speaker 3 (01:25):
Yeah, it's actually really funny.
This is going to sound like Ihad a little bit of a crystal
ball, but this is humble as muchas it is.
This is a true story.
The first day that I got myreal estate license, I got
invited to a big globalconference where it's one of the
rah-rah sessions.
Get 20,000, 30,000 agents readyto go.
(01:47):
I'm not going to name the brandthat I hung my license under,
but we went to this bigconference and, mind you, I have
not sold a house yet.
I have knocked on maybe threedoors and am just cutting my
teeth.
And I'm sitting here listeningto the owner of this real estate
brokerage talk about protectingdata from Zillow and Redfin and
(02:10):
some of the online applicationsthat a lot of consumers back in
2017 were using primarily andstill do today.
And he's going on this rah-rahchant about how we need to
protect our data and it's theagent's data that's important
and we're the ones that arespending the money to collect
these leads and build thesebusinesses around Zillow
applications and everything inbetween.
(02:31):
And then he turns around and hesays so I protected your data
by building our own internalapplication to be able to
collect all this resources andinformation to be able to sell
to your clients and thenimmediately to answer the
question.
It dawned on me in that moment.
I looked to the guy next to mehaven't sold a house yet and I
go.
He's doing the exact same thingthat he's telling us we
shouldn't be doing.
(02:51):
We don't own that application.
And that's really when itdawned on me.
Even after several years, afterbeing a real estate agent and
finding a successful career, Ialways knew that I wanted to get
into technology because I knewthat's where the gold was, was
in the data and being able toreally provide products and
solutions at scale.
And that was really when thelight bulb clicked for me.
(03:13):
It took me a few years becausethe money was good and I enjoyed
it, but I got out as fast as Icould.
Speaker 1 (03:18):
Well, I'll pass it over to Eric.
On that note, Eric, you justcompleted.
Can we talk about the tabletop?
You just completed?
Speaker 4 (03:24):
Yeah, we did a tabletop for a school district
recently and it had some prettygood findings of the respective
teams needed to do to cometogether in the event of an
actual cyber incident, how theywould work together and some of
(03:50):
the things that maybe weren'treally apparent, where you had
some single points within thecommunication chain.
You know, maybe they weren'treally realized until it's like,
wow, this person is doing allof these things.
In the event of a real worldscenario where you're going to
be on multiple bridges, yourphone's going to be banging off
(04:13):
the hook, you're going to begetting text messages, people
are going to be showing up inyour queue, the planning goes
out the window when the shotsget fired right or some
derivative therein.
But the tabletop really bringsthat to light and it does allow
organizations to drill and trainand then if there's an area
where they need to get a littlebit more focused or specific,
(04:34):
they can, and then they can gowork on plans.
They can bring in people ifthey need to, to really iron out
some of those areas that mayfall down in the event of a
real-life scenario and hopefullythey never have a real-life
scenario, but I'm sure as you'vedelivered these, you see that
aha moment and it's kind of funto be there with an organization
(04:57):
as they're going through thatexperience.
Speaker 3 (04:59):
Yeah, I mean, I couldn't agree more.
We've never.
Whether it's a practitionerthat uses our product to
facilitate these as a thirdparty VC so our MSP or whether
it's a direct organization thatis using our solution to scale
or augment you know theircurrent tabletop program.
We've never heard anybody, toyour point, walk out of one of
these engagements and go, wow,that sucked and I didn't find
(05:21):
any value out of it, right?
But I think you really nailedit and you know.
I think you might have said thatNick had a comment about this,
but you know, a plan is only asgood as you can implement it.
And you know what we reallypride ourselves in is because
most of these organizations,whether it's a school or a
manufacturing firm or even atechnology company, most aren't
(05:43):
even testing these plans.
They have a plan in place butthey're not actually testing
them.
And if they are, it's so rarethat our job is to come in and
break these plans, like, at theend of the day, we want to break
them so they can rebuild themand bring it back stronger.
Speaker 2 (05:58):
The culture that we've been seeing with a lot of
our clients is they maybe knowwhat a tabletop exercise is but
they've never participated andthey might have a lot of
branches underneath theirorganization that would funnel
in or have you know to do workif something happened.
Real life happened and I thinkthe quote that Eric was talking
about is something I've saidmany times is when I was in the
military I had a leader tell methe more we sweat in peace, the
(06:21):
less we bleed in war.
So this is directly.
You know I mean differentsituations, right, but if we
plug that into a tabletopexercise, we can show people how
important it is that.
You know, maybe something'sgoing to be pretty clunky, no
matter what right People are.
You're going to be in somescramble effect, no matter what
if it's real life.
But we can take the stress outof it here, slow it down, have
(06:45):
that teaching moment so we canbetter prepare them when they're
actually in.
You know, lack of better termconflict.
Speaker 1 (06:52):
Yeah that's exactly right.
Speaker 3 (06:53):
You definitely want to exercise and practice these
engagements before you get intothem.
You sound like a baseball fan.
You know it'd be like meplaying college baseball and
never taking batting practice upuntil game day.
Speaker 2 (07:03):
You know, it's just no way you're making contact.
Speaker 3 (07:06):
Yeah, it seems odd in that, in that context, but
that's how a lot oforganizations are running.
You know they have, they've gotthese plans on paper and you
know to your point when, when,when the proverbial stuff hits
the fan, you know they'rebleeding, oftentimes because
they've never actually taken aswing or the best they can do
right now is they've got a calltree, right People, they're
(07:27):
going to start calling, butnobody has action items.
Speaker 2 (07:31):
Who's doing what right?
We should be on the same sheetof music before something even
happens, You're nailing it.
Speaker 1 (07:34):
It sounds like by Eric's description.
It kind of gets into that highschool project zone where
there's one person kind of doingmost of the work and everyone
else is kind of waiting for acue.
What was the aha moment for you, cody, that made you realize
this was an issue that you knowyou could help identify and
maybe help people solve?
Speaker 3 (07:53):
At my co-founderized previous firm, we were working
with audit and regulatorycompliance software, so we were
seeing a lot of SOC 2 prep, alot of CMMC and FedRAMP and as
we started to discover moreconversations with enterprise
and men, enterpriseorganizations, predominantly
CISOs and the B-CISOs that weresupporting those efforts, we
were having a lot of discussionaround how do I actually know
(08:15):
that my organization and thepeople that are in charge of
these playbooks and, mind you,these are enterprise
organizations that arerepetitiously testing these
tabletops they do know what theyare.
They are running these on areoccurring basis, as much as
they can.
But they started to ask thequestion how do I measure this
stuff?
First and foremost, and that wasa big aha moment for us.
(08:37):
At first we thought we can buildsome KPIs and some measurable
benchmarking for enterpriseCISOs and VCISOs to be able to
say, rather than just goingthrough a communication exercise
in a tabletop, how do I measurethe projected outcomes that
this might impact and what arethe actions that one might take
(08:57):
place?
And to answer your questionspecifically, our experience in
audit and regulatory compliance,we had a little bit of a knack
for thinking about how controlsaligned to audit evidence,
evidence preparation.
We looked at the samerelationship between the actions
that an individual third party,supplier or a system might take
(09:18):
within an organization in thesame breath is in the event of
an incident, as a control alignsto an audit for preparation, we
want the actions to align tothe coverage for an organization
in the event of an incident.
And that was our aha moment andyou know a few discussions with
some some really well-respectedCISOs and information security
(09:40):
leaders at various organizationsin technology and outside.
We just said you know, this isa problem that we can bring
automation to and reallystreamline for these guys, that
they've never had before andit's just been gangbusters.
It's been fun.
Speaker 4 (09:54):
Cody, what are you doing with AI in your platform?
Are you using it at all?
Have you brought it in forrole-playing scenarios or kind
of?
Where are you on that journey?
Speaker 3 (10:06):
yeah, um, this is a great opportunity to to maybe
give a little sneak peek on ouron our newest product that we're
calling pulse right now.
Um, to answer your question,we've always had generative ai
as an element to generate someof the textual script and story
lines that that typically comewith the creation and
facilitation of these exercises.
(10:26):
That's been the bread andbutter with our legacy products
since the beginning.
I think that one would findthat the common individual
that's organizing tabletopexercises, designing and
facilitating these, the commonpain point that they're having,
is that creativity.
On the front end, I can goonline to an ISACA template, buy
(10:47):
a ransomware scenario and thenswap out the name of the company
and call it a day.
Or I can use generative AIthrough a solution like Opsbook,
which, as far as we're aware of, is the only solution that's
doing this throughout thetotality, where I can have a
subjective scenario that'screated in minutes that is
entirely relevant to the contextof my organization.
(11:10):
So we've definitely usedgenerative AI throughout all of
it, but we've also built anontology on top of that to make
it super, super, exerciseresilience focused.
We want to be able to create anontology on top of that
generative AI to say, hey, ifyou are a manufacturing firm, a
supply chain distributionfacility, if you're an energy
(11:32):
company that needs to supplyenergy to individuals and
consumers, how can we creategenerative AI driven tabletop
exercises but have an ontologythat makes it primarily focused
for our specific industry needsand our specific metrics that we
need to measure, to executethat coverage that we need to
execute?
So, to answer your question,generative AI is entirely
(11:55):
throughout the totality of theproduct.
Speaker 1 (11:57):
I like that vocab word, ontology.
I had to look that up reallyquick.
That's a good one.
Speaker 3 (12:03):
Well, I didn't know until 30 seconds ago, just
disclaimer.
Speaker 1 (12:07):
I had to look it up so that one hasn't come up in my
research, but I was hoping wecould get into some stories.
I know, eric, I'll give you achance to think about it, but I
asked this question to Cody.
But I'd love to hear you knowabout some of the actual
tabletop exercises you've doneand some of the findings maybe
that have come out of it.
You shared with me that yourecently did one with 70
(12:28):
participants and 40 injects.
Was it what chaos or clarityemerged from that process, or is
there anything that surprisedyou or might be fun to share?
Speaker 3 (12:38):
Yeah, from that particular engagement.
I think the exciting thing forus as an organization was seeing
how many people were involvedin that.
We had three differentcontinents that were included in
that and so it wasgeographically dispersed.
So that was exciting for us tosee, historically, what's always
(12:58):
been a face-to-face interactionor a smaller Zoom engagement.
With the technology that we putin place in OpsBook we've
actually been able to scale that.
So that was just kind of thefirst aha moment of like wow,
this is really applicable forlarger engagements.
We now give intercontinentalorganizations accessibility to
tabletops like they've never hadbefore, because in a Zoom call
(13:20):
you can't collect theinformation, you can't make it
subjective, you can't go througha PowerPoint deck and also see
the level of engagement thatwe've been able to create within
the conduction portion of ourproduct.
So that was a really excitingmoment for us in that particular
tabletop.
But, to be honest with you, thismight not be the answer that
you want to hear, but we reallysolve for a lot of the mundane
(13:43):
stuff that enterpriseorganizations, you know, don't
want to necessarily have tomanually go through.
So in an organization like thatthat's large, they're running
so many exercises that areoutside of the realm of just
cybersecurity.
It falls into IT process.
It falls into supply chaindistribution.
You know pathways.
(14:04):
It falls into emergencyresponse plans for a natural
disaster or an earthquake.
Work with financialinstitutions that run exercises
about.
You know, unfortunately, crisisand shooter scenarios, you know
.
You mentioned, obviously, ericrunning one with a school.
You know these are real worldscenarios that we just get
really excited about the optionto take not crazy scenarios, but
(14:29):
the scenarios that need to berun, that are actually going to
happen in the real world, andbring those to scale.
Speaker 2 (14:35):
Cody during that big 40 inject tabletop.
You guys did.
How long does that tabletoplast?
Speaker 3 (14:43):
Yeah, we chuckle because that particular one was
about six hours and I don't knowa person in this world that
would want to be in a tabletopfor six hours.
But uh, we built some coolfunctionality to where you can
break these things up and so, uh, you know those, those 40
injects to your point.
You know, think of those as uhkind of a chronological or a
(15:03):
sequential timeline in the eventsimulated event.
So you're not always testingeveryone at once in those long
events, so you might have fouror five injects that go against
one particular department, fouror five that go to the next.
You can kind of put some peopleon ice and let them go take a
break and do some things.
But it went quite smoothly,guided by the product you took
(15:25):
started answering my nextquestion.
Speaker 2 (15:26):
I was going to ask if you guys are breaking that up
by maybe by department or groupof people, letting some, like
you said, take a break.
But yeah, that's awesome thatyou guys can do that with the
tool.
Speaker 3 (15:37):
Yeah, one of the biggest things that we've found.
And I don't know how manytabletops you guys have done,
but in almost all of them wealways find that there tends to
be a rabbit hole.
Right, there's either somebodythat just got hired on as a part
of their onboarding theyhaven't gone through their
training, they're a little bitinfant in their role or somebody
super experienced and says I'vebeen here for 20 years, I got
(15:58):
all the tribal language down.
That's not how we do thingshere, and in both cases there
tends to be this rabbit hole ofdialogue.
And so, to your point, wecreated this concept of what we
call a branch, which is think ofit just like a breakout room in
the product, and so we've givenpoints for those discussions to
be had, the notes to be loggedand then also to be brought back
(16:19):
to the guide rails of what theexercise was meant to be.
Had the notes to be logged andthen also to be brought back to
the guide rails of what theexercise was meant to be.
So we've thought of a fewlittle fun things like that in
the product.
Speaker 2 (16:28):
I just picked up on one thing you said.
You said tribal knowledge.
That is something we, I think,are always going to see, but
specifically we were doing notnecessarily a tabletop exercise,
but we were doing some planningbecause somebody was going to
retire at a client we wereworking with.
This is like a year or two agonow and he had been there for it
was like 30 years and he was aleader of the IT space there or
(16:50):
the IT group, and it was alltribal knowledge.
There was virtually nodocumentation and that was a lot
of our conversation.
Was you know, catastrophicevent?
Whatever it is, you know?
This is we're getting to thatpoint.
Now if he's leaving, what areyou guys going to do if he's
gone tomorrow?
So that planning getting readyfor a tabletop exercise for a
(17:11):
lot of organizations just can bea good exercise before you even
get there, because they're youknow.
When you're telling them, ok,we're doing a tabletop in a week
or two, whatever the timeframeis, they're subconsciously at
least getting ready right orthey're preparing.
So that can be helpful.
Speaker 3 (17:26):
Yeah, you're nailing it.
I mean, at the end of the day,that travel knowledge from that
individual that's been there for30 years.
A lot of our customers areusing us to replicate that
throughout the playbooks thatthey need to continue after that
person retires.
So the backend element toOpsBook is not only the creation
of facilitation, but a hugevalue add is what we like to
(17:47):
call the after action reviewreporting template, and one of
the parts of that is that thiscan act as a source of truth or
a repository for people thatdon't know what am I going to do
in the event of this scenarioof this guy that's been here for
30 years that I've never youknow, I've never done this
before.
They can refer back to thoseafter action reviews, quickly
(18:08):
filter through them and seeexactly what he would have done
in that instance.
Speaker 2 (18:12):
So this is after the tabletop's been done.
Is there a timeframe that theycan go back to this information
or are they extracting it out ofops book and it's a takeaway
where they can use this ashomework or, you know, every
year, every six months, whateverit is, they can go back and
review, make sure their house isin order yeah, it's all the
above.
Speaker 3 (18:30):
To be honest with you , nick, some of our customers
use it as a source of truth tojust refer back to for
continuous training.
Or even just look at theiterations of playbook changes
over time Like, hey, a year agowe found these gaps, we've
shored them up.
Now we've tested them again.
Now let's look at the newplaybook.
We've had organizations use itfor training perspective, where
(18:50):
they take an app lecture reviewand then they recreate a
scenario to use it for trainingand onboarding to be able to
effectively test people in thoseroles.
They can also use itinstantaneously after the
exercise for audit andregulatory compliance purposes.
We come from that background,right.
So we have that in mind to say,hey, if you're going to be
going for a CMMC or a FedRAMP oreven an ISO 27001 that requires
(19:15):
these things, how can we push areally, really specific example
of this evidentiary item orrequirement into a GRC product?
And so we've got, we've donethat as well.
Speaker 2 (19:25):
Yeah, I think you're getting at.
The same thing I've beenthinking about is, you know, for
doing a tabletop exercise today, we don't want it to just be a
two to six hour event.
You know this should be anongoing right.
Once you're done here, itshouldn't just be a check in the
box hey, we're done, we did ourtabletop.
Maybe it's for compliance, ormaybe it's their cyber insurance
says they need to do a tabletopexercise.
There could be many differentreasons they're doing it, but
(19:47):
you know we really want it to beokay.
You learned this today.
Now let's apply it and keepreviewing to make sure we get
that into our tribal knowledgeor our standards.
Speaker 3 (19:57):
Yeah, we, we like to.
We like to coin a phrase onthat.
You know we help teams turndecision-making into muscle, not
a memo, and that's what we liketo say about the reoccurring
idea of tabletops beingsomething that's accessible and
reoccurring.
Speaker 1 (20:13):
That's awesome.
So not to over-egg the puddinghere, but if someone's you know
looking to maybe work for ITAutolabs or thinking about, you
know, opsbook or implementingsome software like this, I like
to kind of give them a highlevel overview and maybe some
info.
Information from Eric as a CISO.
You know your experiencerunning these things and you
(20:34):
know I'm always looking for thejuicy stories right or for
outlying kind of events.
But maybe you could shed somelight on those types of things
or how you prepare or get peopleto do their homework once the
tabletop is concluded.
Speaker 4 (20:47):
Yeah, sure.
So I think it's a couple ofthings on the tabletop side and
all of the real world scenariosthat I've been involved with and
have triaged, either as anincident person responsible for
the incident, the actualincident response or, you know,
(21:07):
as the CISO.
They've never actually marriedto a tabletop experience, in
that the real world scenario isgoing to be pretty mundane At
least the ones that I've beeninvolved in typically comes in
(21:28):
over fishing or you get an alertthat something doesn't look
right and there's activity thatis not consistent with usual
behavior, and then you go andyou find out that yes, you know
we've had a breach, we've had anincident in the environment and
now we have to recover fromthat incident.
(21:51):
And you know you go intocontainment and you're going
through all of the steps thatyou would in a tabletop around a
ratification and the aftermaththere of recovery and where it
differs in the real world is theamount of distractions and
(22:14):
meetings and just follow up anaftermath that you're pulled
into in the course of resolvingwhatever the issue is.
Because there's, if you'reworking in an organization
that's got regulated data,you're trying to figure out well
how much data is potentiallyexposed and all of the different
(22:35):
third parties that you'reworking with to resolve the
incident and you can, in a safespace not in the heat of the
moment talk about well, okay,yeah, we can absolutely do a
recovery.
What happens if we recover andthe infection is still in place
and the threat actors are ableto turn that up?
That's easy to do and welcomedin a tabletop exercise where you
(23:00):
can then go down differentderivatives of okay, yep, we're
going to restore it.
And then, if it's still there,what do we do?
Or no, we're not going torestore, we're going to rebuild.
You know whatever thosescenarios are.
But in the aftermath, when youhave a, say, a billion dollar
organization that has livecustomers that either can't get
(23:21):
ahold of somebody or they'rebanging away on the phones, you
have news reporters, you haveall of this commotion happening
and you have the leadership ofthe organization saying you need
to recover, that we have to beup.
There's swearing, there's fistspounding on the table.
We got to get that back up.
We got to get it back up rightnow.
(23:42):
And if you go through thatexercise and you drill that over
and over again and you havethat conversation with those
leaders, like, yeah, you'regoing to be hot under the collar
.
The board's going to be callingyou.
You're going to have to makethat decision.
You're going to have to pushback when I tell you, no, we
cannot restore.
You're going to have to trustme, and here's why.
But in the heat of the moment,if I'm telling you, you have to
(24:04):
trust me, and here's why you maynot hear that unless you've
seen all of the steps that theIT organization has already gone
through, and why you can trustme when I'm telling you that.
So that's why I really love thetabletop exercises, because you
can hit pause on any one of thespecific scenarios and you can
dive in and you can reallyreplicate what that's going to
(24:27):
look like in a real environment.
Speaker 3 (24:29):
When we kick things off.
We originally identified as anautomated tabletop solution for
that exact reason you know someof the generative AI, you know.
Just to iterate and echo whatyou're saying, we've recently
adapted more of anidentification of exercise
resilience, because the wholepoint of these tabletop
exercises is a proactive and apracticed and a learned behavior
(24:54):
leading up to the event.
But you know you nailed it Likethe real world scenario on the
response side is what's mostimportant is can we accessibly
access that coverage and do weknow how to execute it in the
event of a real world, uh,incident?
And you know I mentioned kindof teasing our newest product.
You know that we're callingpulse and I mentioned an action
(25:15):
catalog and that's exactly whatwe're trying to do, eric, is
we're trying to.
We're trying to associate theactions that one might have
practiced in a tabletop exerciseand make those readily
available in the event of anincident.
So if we can categorically takepracticed, simulated behaviors,
responses, actions, systems,suppliers, and almost build in
(25:41):
conjunction with that ontologylike a master spreadsheet of
approved actions that anorganization might take, and I'm
talking down to the granularlevel grab the wheelbarrow,
throw the servers on there andgo out in the earthquake.
If we can categorically havethose things organized, then
when it does come time torespond, you don't have to think
(26:02):
twice, even though you havepracticed it.
You've got those readilyavailable at your fingertips and
that's what we're really goingwith our newest product, Ups.
Speaker 4 (26:08):
But Pulse and I think that tabletop exercise and
maybe that's what you're talkingabout in your new product set
where people can simulate thosethings, where they can go and
say, ok, you know, this is whatis happening.
Here is the scenario.
Let's just iterate on.
What are all of the things thatwe're going to face?
(26:29):
What are they going to tell usthat we can do or you know we
can't do?
Let's bring in some of thosethird party, realistic, real
world scenarios where we have aserious Microsoft issue.
We're going to call Microsoft,we're going to open a SEV1.
Microsoft's not going to jumpon the phone right then and
solve our issue right.
It's going to get passed around, it's going to get escalated.
(26:51):
They're going to want to knowif we have 24-7 support and all
of this other nonsense beforethey even engage with us on this
SEV1.
So we have to really bethinking of, in the heat of the
moment, how is a scenario wherewe need third party help?
Can we even get it?
And if we can't, what are someof the things that we could be
(27:13):
thinking about doing rather thanjust waiting and saying oh, we
have a SEV1 with Microsoft, youknow they're going to come in
and solve all of our problemswhen, in practice, we know that
that's far from the case Exactly, and that's really why we're
going to come in and solve allof our problems when, in
practice, we know that that'sfar from the case Exactly, and
that's really why we're tryingto spin tabletops on his head a
little bit with our newestproduct.
Speaker 3 (27:29):
So imagine having a playbook that's prescribed in
place of approved actions thatyou, as a CISO, allows your
organization or third parties onyour behalf to execute in the
event of an incident.
We've got softwares in placefor these things that can
automatically make decisions andcreate tickets and execute
accordingly.
We've got individuals.
You know there's organizationalroles and responsibilities that
(27:51):
fall into that.
It's a fairly dynamic process,but if you can categorically
have those identified, ratherthan starting with the tabletop
and then moving to approvedactions, we're flipping it on
its head with the newest product, which is let's start with how
you'd actually solve the problem.
Let's start with the movingtruck and throwing the servers
in there, and then we canmitigate the gaps after.
(28:12):
And by doing that we are nowusing, with the help of AI built
in the product and the ontologythat makes it subjective, we
are now able to take a scenarioand test it hundreds of times
with the software in betweenthese engagements.
So when you show up to atabletop exercise that might be
(28:32):
testing a business continuityplan annually or an emergency
response plan or an incidentresponse plan.
Every 90 days you're showing upwith the most updated version
of that action catalog ofsolutions that you might take on
behalf of your organization andif there are gaps or
fine-tuning that you need to doas a mid-enterprise or an
enterprise org, you canbasically say that to a degree
(28:56):
of certainty.
We've tested this a hundredsimulated times and now we
brought it to the finish line onexactly how we want to perform
on it.
So you are now far moreprepared than just running this
once every 90 days or once every12 months.
Speaker 1 (29:09):
I can really relate to that.
Growing up in a remote part ofMinnesota, the North Shore kind
of had to be thinking on ourfeet and finding ways to solve
problems with what we had theresources available to us in
that moment because we couldn'tjust order something on Amazon
and is available to us in thatmoment because we couldn't just
order something on Amazon and Iwas two hours away from a bigger
city.
So I think that is kind oftranslated into my professional
(29:30):
life.
But also I think that's how weview things at IT Audit Labs too
.
It's just like an overallculture of getting things done
and it sounds like that appliesto Opsbook as well and you can
kind of bake some of that stuffinto the inputs, am I correct?
Speaker 3 (29:47):
That's absolutely right.
Yeah, and the beauty of it isthat if we're partnering with an
IT audit labs or a third partyconsultant or a VC, so we've
really designed the system tokeep that stuff in mind.
So whether it's an organizationor a consultant in a smaller
organization, it's really comingin to help.
You know, those inputs areimportant for the subject matter
(30:12):
experts, but if we can testthem at scale, even for smaller
organizations, then we're nowdecreasing response times.
You know, we're strengtheningteam cohesion at smaller
organizations with small teams.
We're strengthening teamcohesion at smaller
organizations with small teamsand we're streamlining the
relationship for third partieslike IT Audit Labs or
consultants on the side toeffectively come in and say I
can now make your smallerorganization operate like an
(30:35):
enterprise organization with atool like this in place.
Speaker 1 (30:42):
I know we work a lot with the sled in that space,
sled space and I just wascurious if Eric had kind of a
go-to like monkey wrench youmight throw into a tabletop
exercise to kind of get thepeople to think on their feet or
create a culture of problemsolving.
And then maybe we could getCody to even follow that up for
how the ops book might approachsomething like that.
As we talk about macro andmicro, is there kind of a go-to
(31:03):
monkey wrench you like to throwinto a pen test or, sorry, into
a tabletop exercise that kind ofgets people to think on their
feet?
Speaker 4 (31:10):
We like to engage with the person who is
sponsoring the exercise andreally understand what it is
that they want to get out ofthat exercise.
Right, is it more on thetechnology side?
Is it more on the leadershipside?
Is it more on the leadershipside?
Is it more just on thevisibility side?
So certainly we'll come up witha scenario based on what that
(31:32):
person is looking for.
But the question that we alwayslike to ask maybe it's the the
monkey wrench is who gets topush the button Right and at the
end of the day, when the when,when the scenario is going on or
the real life scenario ishappening and there's something
(31:53):
that will be impactful to thebusiness, where you're maybe
taking some some very criticalsystems offline to resolve an
issue, or you know you have tomake that call to do something
that is largely impactful, whois the one person that can make
(32:16):
that decision?
And you know, sometimes we hearit's well, it's like a group of
people and we really try todrive down, like who is going to
push that button when the timecomes?
And that's hard fororganizations to answer a lot of
the time.
Speaker 3 (32:34):
The one that seems to be tested the most is obviously
ransomware, but what they don'tinclude in that typically is
insider threat.
They don't include in thattypically is insider threat.
I mean, it's just, it's one ofthe ones that every time we see
an inject in a tabletop exercisethrown in with an insider
threat, nobody seems to learnhow to respond or know how to
respond, and it's almost likethere's a cultural offense to.
(32:58):
Even in a hypotheticalsimulated scenario in ops book,
how could I possibly imaginethat John who's in the cubicle
next to me could possibly be thereason that we're being
attacked right now?
That's always a fun one tothrow at people.
The responses seem to go youknow just every which way under
Sunday, but it also is commonlyidentified as a gap and a great
(33:21):
thing to test whether it's aransomware scenario or just an
insider threat by itself.
The second one that's alwaysbeen fun and I think this is
relatively common knowledge inall of cybersecurity and even
physical security is if you'regetting attacked by a threat
vector or a hacker.
For example, they've had accessto your system far longer than
(33:42):
yesterday, for example, they'vehad access to your system far
longer than yesterday, and whenwe see curveballs or injects
that are thrown in thosescenarios around, what happens
when you have updated playbooks,but so does the hacker?
They have access to yourplaybooks.
How do you respond when yourplaybooks are now null and void
in totality?
(34:02):
You cannot use any of them, andso when we throw that inject, I
think it dawns on a lot ofmature organizations that even
are testing these exercises on areoccurring basis.
They go oh crap, we don't havea plan B for these scenarios.
And if we do follow the scriptof these playbooks or these run
books in the event of anincident, he knows my moves
(34:24):
minutes, hours before I take itright.
So I think having somecontingency plans in place is
always a fun one for us todiscover in some of these
exercises.
Speaker 4 (34:32):
I like that you brought that up, cody, because
it really articulates howorganizations should be treating
and drilling the playbookswhere it's not really a script,
and I think we saw that withsome of the more popular tv
shows um in in the late 90s,like I think it was maybe
friends and and seinfeld.
(34:55):
I think those are the twoexamples where they weren't
necessarily scripted, where thethe scene was written and the
actors were given the generaldescription of the scene, but
the actual words that they usedwere open for the actors to
interpret, to convey the messageof the scene, versus reading
(35:18):
and memorizing verbatim.
Here is the exact script.
I'm going to say this.
This person is going to saythat verbatim.
Here is the exact script.
I'm going to say this.
This person's going to say thatBecause in a real world
scenario you can never drillexactly what's going to happen.
You mentioned the insiderthreat.
There could be an insiderthreat piece.
There could be a person who iscritical to the success chain,
is sick or out on vacation orwhatever that is.
(35:40):
But if the organization isdrilling the actual context and
not exactly the steps in theplaybook, I think they'll be
more successful, and it soundslike that's what OpsBook can
help them do.
Speaker 3 (35:53):
Yeah, I'll make two quick comments on that.
I'm a baseball guy, love thecaptain.
I talked to a guy that's a BoSox fan today but I love Derek
Jeter and I'm not a diehard NewYork guy but I love the cap and
in his retirement he really saidit best for a baseball analogy.
That I think aligns really wellwith cybersecurity and some of
these resilience plans.
(36:14):
As he said, I've simulatedevery potential play that might
present itself to me before thepitch is ever thrown potential
play that might present itselfto me before the pitch is ever
thrown and I think we really tryand take that mentality with
the action catalog and our newproduct to be able to say why
should tabletops be a staticengagement?
(36:35):
Why should they not beconstantly simulated in real
time for enterpriseorganizations?
So in the event that an injectdoes happen, where somebody has
access to my most updatedplaybook that I'm housing in my
GRC or housing in a third-partysystem, I can log into OpsBook
and immediately have the mostiterative updated version of the
solutions, the steps and theactions that I would take place
(36:56):
in the event of this breach.
Because my process, my people,my onboarding, my ticketing
system everything's changed inthe last 14 days anyways.
So why would I be using a 14day old playbook when I can use
a two hour old one?
Why go to battle with medievalarmor and a long sword when
you're playing a war that'sfought with drones?
You know what I mean.
(37:16):
You really got to step up tothe modern day when it comes to
resilience, and that's whatwe're trying to do with our
newest product, for sure.
Speaker 1 (37:24):
Well, thank you so much for your time today, cody.
We've been talking to CodySullivan from OpsBook.
My name is Joshua Schmidt,co-host and producer.
I've been joined by Eric Brownand Nick Mellon from IT Audit
Labs and have a great day.
We publish episodes every otherMonday.
We're on Spotify, youtube,amazon, apple you name it.
So wherever you find yourpodcast, give us a subscribe, a
follow and a comment and areview.
Speaker 4 (37:48):
If you have the time.
Thanks a lot for listening.
You have been listening to theaudit presented by IT Audit Labs
.
We are experts at assessingrisk and compliance, while
providing administrative andtechnical controls to improve
our clients' data security.
Our threat assessments find thesoft spots before the bad guys
do, identifying likelihood andimpact or all.
Our security controlassessments rank the level of
maturity relative to the size ofyour organization, thanks to
(38:12):
our devoted listeners andfollowers, as well as our
producer, joshua J Schmidt, andour audio video editor, cameron
Hill.
Cameron Hill, you can stay upto date on the latest
cybersecurity topics by givingus a like and a follow on our
socials and subscribing to thispodcast on Apple, spotify or
wherever you source yoursecurity content.
Popular Podcasts
United States of Kennedy
United States of Kennedy is a podcast about our cultural fascination with the Kennedy dynasty. Every week, hosts Lyra Smith and George Civeris go into one aspect of the Kennedy story.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Bookmarked by Reese's Book Club
Welcome to Bookmarked by Reese’s Book Club — the podcast where great stories, bold women, and irresistible conversations collide! Hosted by award-winning journalist Danielle Robay, each week new episodes balance thoughtful literary insight with the fervor of buzzy book trends, pop culture and more. Bookmarked brings together celebrities, tastemakers, influencers and authors from Reese's Book Club and beyond to share stories that transcend the page. Pull up a chair. You’re not just listening — you’re part of the conversation.