August 29, 2023 • 33 mins
The Security Table team dialogues about the importance of data and metrics in understanding and communicating risk. After Matt defines ROI, Izar emphasizes that while data is crucial, it doesn't always come in numerical form. Instead, risk can be expressed in various ways, such as trends, and doesn't necessarily need to be quantified in traditional terms. Chris stresses that executives need tangible metrics and data to make informed decisions, especially when communicating with legal teams and other stakeholders.
They then talk about visibility and understanding the attack surface. Izar explains that the attack surface represents an organization's exposure to potential threats. The goal is to provide a comprehensive picture of the organization's vulnerabilities and the measures taken to address them. Instead of inundating executives with technical reports, Izar suggests telling a story that conveys the essence of the risks and the steps taken to mitigate them. Chris, however, emphasizes the importance of concrete data and the challenges executives can face in understanding technical nuances.
Lastly, the dialogue touches upon the real-world implications of threat modeling and its ROI. Matt Coles highlights the potential legal and business repercussions if things go awry. The discussion underscores the evolutionary nature of threat modeling, with Izar noting that while one might start with limited expertise, continuous learning and adaptation lead to improvement over time. The overarching theme is the balance between technical details and business-oriented communication, ensuring that executives understand the value and impact of threat modeling initiatives.
Links referenced:
- US Executive Order 14028 on cybersecurity - https://www.federalregister.gov/documents/2021/05/17/2021-10460/improving-the-nations-cybersecurity
- CISA, Secure by Design, Secure by Default - https://www.cisa.gov/securebydesign
- Secure Software Development Framework (SSDF) from NIST - https://csrc.nist.gov/Projects/ssdf
FOLLOW OUR SOCIAL MEDIA:
➜Twitter: @SecTablePodcast
➜LinkedIn: The Security Table Podcast
➜YouTube: The Security Table YouTube Channel
Thanks for Listening!
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Chris Romeo (00:09):
Hey folks.
Welcome to another episode ofthe Security Table.
This is Chris Romeo, joined bymy good friends, Izar Tarandach
and Jerry Garcia.
No, no, no.
Sorry.
Sorry.
I forgot I was a little, little.
A little confused for a second,but it's actually Matt Cole, not
Jerry,
Izar Tarandach (00:28):
just because he looks a bit, a bit hazy.
Chris Romeo (00:32):
Just look, maybe he could sing a couple of, uh, a
couple of bars for us, maybe?
Matt Coles (00:37):
Absolutely not.
Chris Romeo (00:38):
Absolutely not.
Thought we might get a, youknow, truckers or, you know,
lemme see.
Something Magnolia Sweet Magno.
I don't remember.
Matt Coles (00:48):
I have no idea what what you're talking about.
Chris Romeo (00:50):
Grateful Dead songs, but, so yes.
Matt Coles (00:55):
College was a haze because of, uh, great Grateful
dead song.
Chris Romeo (00:59):
There you go.
You're not, you're not oldenough for that to line up,
unfortunately.
Matt Coles (01:04):
Tell that to the rest of my floor in college.
Chris Romeo (01:07):
So we're gonna continue our.
Jaunt into the world of threatmodeling.
So in our previous episode, wehad a great conversation with
Jim Manco about where he sits inthe world of threat modeling,
and we heard about how he, uh,where, where his ideas were kind
of coming from.
I thought that was prettyeye-opening.
His view of consultants and howthey abused threat modeling in
(01:28):
the early years of AppSec, Ithought was, uh, was a good
perspective.
It was one that I hadn't.
Necessarily seen firsthand beinginside a product company and,
and not having consultants thatwere being, that were doing the
threat modeling for us.
I thought that was a reallyinteresting viewpoint, but we
want to talk about the return oninvestment for threat modeling.
(01:49):
And so I thought we'd start by,let's define return on
investment just for those peoplethat maybe, um, aren't as deep
into the business side of.
Running a company or being apart of a big company where
return on investment's a bigdeal.
Matt, why don't you kick thatone off for us?
What is return on investment?
'cause I know Izar likes to justcome in and say same or
(02:12):
basically flip around what yousaid completely.
So, Matt, return on investment.
What is it?
Matt Coles (02:18):
Um, so actually, uh, I'm gonna defer back to you
because I think you've got the,the, the broader business sense
here.
Uh, so I.
I mean, I know generally whatROI is, right?
It's a common term we usethroughout the security world
and, and obviously, uh, if youhave any of the big name
certifications, ROI must havecome up in those.
But in a, in a bare sense, it's,it's a cost benefit system,
(02:44):
right?
It's, it's a system of how muchmoney do you spend versus what
do you get in return.
There's more to that.
So I wanna throw it back to youactually,'cause I think you have
a good sense of this.
Obviously, you're verysuccessful in, uh, in running a
business.
So you've done ROI and you'veeducated others on how to do ROI
for security.
So let me throw it back to you.
(03:05):
Help us understand what ROI isis here in context.
Chris Romeo (03:08):
Yeah.
I think your, your definition isright on as far as it is, what's
the value that's being generatedfor the dollars for the
resources could be humanresources as well that are being
expended towards a given cause.
And so when I think ROI, I thinkthere's kind of startup ROI
world, and then there's bigcorporate ROI world.
(03:29):
And in the big corporate ROI.
Often executives are looking forsome type of metric of what am I
getting for the investment thatI'm making of dollars in people,
security team.
Executives don't care about howmany alerts are generated, how
(03:50):
many SAST findings there were,how many, uh, vulnerable third
party packages we have.
Executives don't think likethat.
Executives run businesses andreturn on investment is as
simple as saying, you're askingme Izar to spend$500,000 and
expend 500 people each a week oftheir time.
What am I getting?
(04:11):
What are you providing me as theresult of me making this
investment?
And so it's really a businesscase conversation.
It's, it's, are you providing meenough value as an executive
where I can say, okay, I'llinvest that money because I see
what I'm gonna get out of it.
That's really return oninvestment in, in the startup
world, it's, it's a lot smaller'cause there's not as many
(04:31):
people talking about it, but itis still important.
Like it, when I was runningSecurity Journey for example, we
would be often have a decisionin front of us and it would have
a dollar cost and we'd have to,we'd have to wrestle with as a
small group of executives tosay, does that make sense?
Should we move forward with thisor do we hold off and wait
another quarter?
(04:51):
Or, you know, before we investthese dollars.
So that's kind of my take onreturn on investment.
Hopefully it, it helped Matt to,to provide some context from
where I'm coming from.
Matt Coles (04:59):
Oh, absolutely.
Um, so if I could just, Put somethoughts in here and then is
our, I mean, we're good.
We'll just unleash the, unleashthe Kraken Izar uh, so, uh, you
know, just, you just keepdrinking your monster over there
and get, get pumped up for this.
Uh, so if we think about costand cost and benefit, right?
(05:19):
How much we spend, obviously,you know, it's, it's people and
time and te and tools andtechnology to not only do threat
modeling, but also to manage theresults.
Right.
So how many people do you need?
Do you, do you need bring inconsultants?
Are you doing training ahead oftime or does everyone know what
they're doing?
And then can you actuallyexecute?
(05:41):
But I would suggest at least theinitial benefit of threat
modeling is actually in terms ofrevenue protection and, and
preventing rework because theoutcomes of threat modeling are,
uh, issues that you find thatmay influence the design that
(06:01):
need to be fixed.
You fix those upfront, thatimproves your design, but really
what you're doing is preventingvulnerabilities and, and other
things that will appear later inthe development cycle and things
that will get released tocustomers resulting in
effectively revenue protectionas your benefit.
Chris Romeo (06:20):
Can I put my, my executive hat on for a second?
Absolutely, please do.
And we can have a little, we canhave a little, uh, mock kind of
approach to how this mightactually go down in a business.
Because when I hear what,basically what you just said, my
immediate question to you thenis, how are you gonna prove that
to me?
How are you gonna prove thatthere's less rework?
(06:41):
What's your metric gonna bethat's gonna drive this to allow
me to say, okay, now Matt toldme this and this and that, and
things are gonna be better if wemake this investment.
How am I gonna measure you?
What, what number are you gonnaprovide for me that lets me
measure whether you were tellingme the truth when you set this
business case up for?
Matt Coles (06:58):
That's a great question.
I don't have necessarily have agood answer for you.
Izar?
Izar Tarandach (07:06):
Well, in the past five minutes, I have seen
the past 20 years of my life runin front of my, my eyes.
Right?
Matt Coles (07:14):
Are we that old?
my God.
Yeah.
Izar Tarandach (07:17):
Amazing.
Least.
So yeah, that time passes whenyou're having fun uhhuh.
But, uh, you know, the, the, thething is the the, when I was a
young threat modeling padawanand somebody asked me, why
should I do this thing?
The question itself wasunthinkable to me.
Well, what do you mean what,what do you get if you do this?
(07:41):
Don't you realize that if youdon't do this, the whole world
is going going to come crumblingdown and the space time
continuum is going to fold onitself and there's going to be
one big black hole where yourcompany was?
Turns out that I was wrong andthat there are people who will
gladly go about their liveswithout doing that.
(08:04):
And, uh, that conversation withJim, for me, it was enlightening
on, on, on a lot of differentfields, but his insistence on
the wasteful side of threatmodeling showed me a, a couple
of things.
First, that unfortunately Jimneeds to get better informed
(08:25):
about the new ways of doingthreat modeling that have
flourished in the past fewyears.
So where we, we, we haveaddressed a lot of that waste.
But not only that, not not onlythe waste we have, we have
addressed in general.
The effort and what to take outof the, of the, uh, the process.
(08:47):
And, and I think that today it'seasy to say it, it's actually
not easy.
It's, it's clear to say thatthreat modeling has evolved to a
point where we are not onlyidentifying and mitigating and
accepting risk through designweaknesses, we are going way
beyond in terms of understandingwhat are the systems that we are
(09:08):
building, which are forevergetting more and more and more
complicated.
We, we, we are giving, uh, uh,transparency over these things
that we are putting out thereand that are responsible for so
much of, sorry, whatcivilization is doing right?
And we have extended intoprivacy.
We have extended into so manydifferent fields that deal with
(09:28):
this whole experience that welive today.
That the question to me, has tobe turned around and say, how
much are you going to lose ifyou don't?
Especially with us coming to apoint where people are starting
to talk about, again, that wordliability.
When you, when you write, uh, asoftware and you put it out and
(09:50):
the, the, the flag that thethree of us have been bending
about of what's reasonablesecurity, is there anything more
reasonable in terms of securitythan taking time to sit back and
think what could possibly gowrong?
If you have to to discuss, isthis something that I want to do
or not?
I would say that you have asignificantly more deep problem
(10:14):
than what's the ROI on thisthing.
Chris Romeo (10:16):
Yeah, but remember I'm an executive, okay?
And if you tell me that I needto somehow extract the value
myself out of the beauty ofthreat modeling, I'm not gonna
do that.
Because I don't think, I'm not asecurity person.
I'm a business person.
I care about dollars and centsand bottom line, I have an, I
have a number I gotta hit thisquarter to make Wall Street
(10:38):
happy.
Yeah.
So let me, let me add, that'swhat I care about.
So that's, so that's why yougotta tell, you can't, you can't
flip it back on me.
I'm not gonna do it.
I'm just, no way...
Matt Coles (10:46):
can I just add
Chris Romeo (10:46):
that's happening.
Matt Coles (10:47):
I just have one go.
Izar Tarandach (10:49):
Just before we go there, I, I have an
unflinching admiration forpeople who are able to get a pot
of water, throw all the data oftheir business inside, let it
simmer for a while, and come outof something that they call a
threat modeling a threat model.
That's, that's great, especiallyif, uh, tomato paste on it.
(11:11):
But, uh, uh, the, the, the, thepoint is that not everything
translates down to dollars.
At some point we have to moveaway from those numbers and say
it's, it's, it's, it's a qualitything.
It's not a quantity thing.
Chris Romeo (11:24):
Everything boils down to dollars in business, the
executive level.
No, no.
I'm still got my executive haton.
Izar Tarandach (11:31):
Yes.
Matt Coles (11:31):
So let, let, let me introduce it.
Chris Romeo (11:34):
Let Matt, let Matt get in here.
Matt Coles (11:36):
I'm gonna, so I'm gonna, I wanna add something to
the benefit column here and, andI think this will maybe talk a
little bit closer to what you,with your executive hat on is
thinking.
So revenue protection is alittle bit more, is a little
amorphous.
We can talk about things likereduced support calls.
Better or improved customercustomer engagement, improved
(11:57):
customer acceptance.
But Izar, you brought upsomething very particular.
So we've seen the past coupleyears with the, uh, US uh,
executive order aroundcybersecurity and the CISA,
secure by Design, secure bydefault, uh, and the, the SSDF,
uh, you know, secure softwaredevelopment framework from NIST
and, and, and the collection ofagencies across the world that
(12:18):
are adopting those similarpractices.
If we look at threat modeling asa, an aspect of due diligence,
and now I'm gonna bring in thelegal aspect here, due
diligence, the thing that you doto ensure that your software is
free of, of easily discoverable,discoverable vulnerabilities,
and so that you can meetregulatory compliance and legal
(12:41):
expectations and obligations.
Threat modeling becomes anaspect of due diligence.
Due diligence drives regulatorycompliance.
Regulatory compliance means thatyou have revenue protection, and
now you have a dollar and centsdiscussion.
Does that, does that meet yoursmell test there, Mr.
Executive?
Chris Romeo (12:58):
I mean, I think you, I think you're, I think
you're taking me on a, apathway.
I'm, I'm following you on thepathway, but I still need more.
It's still too nebulous to sayrevenue protection and, um, due
diligence and all of thesethings.
They mean stuff, but there's noway I can measure it right?
Now, where, where you were goingwith reduced support calls,
(13:18):
reduced support costs.
Now this is a metric I can lookat.
Because I'm not, we're not gonnaroll threat modeling out in one
week.
We're not gonna say, next weekis threat modeling week.
Get ready everybody.
Everybody's doing nothing else.
We're not doing anything.
We're not building any newfeatures in our company.
All we're doing is threatmodeling, right?
You're gonna roll this out overa period of time, 2, 3, 4
(13:39):
quarters.
If we're talking largeenterprise now, what if we pilot
that as an executive?
I'm gonna say, okay, Matt, Ilove this idea.
I like where you're going here.
Lower support costs and, and,and, and less rework.
Let's do a pilot for a quarterwith a particular business unit
and collect your data and thencome back to me and show me that
(14:02):
you in fact have lower supportcosts and you have less rework
and you, you know, all thosethings you can measure because
I'm not gonna write you a checkfor$5 million on day one.
I'll write you a check for, for$500,000 on day one.
If I believe in the idea and Ithink you got something, but I
can't roll it out enterprisewide without a, without a proof
(14:22):
of concept that shows me yourdata backs up what you were
telling me and Izar's about tofly out his chair.
Izar Tarandach (14:28):
No, but, but, but, but Chris, think about it.
Okay?
What you're telling me is you,Mr.
Executive, are willing to writea check for DAST because DAST
gives you numbers at the end.
Threat modeling doesn't.
Chris Romeo (14:48):
Can I take my hat off for a second and just yell
into the microphone?
No, I mean, I'm, I'm gonna.
Matt Coles (14:53):
You could have chosen of all, any other letter,
but you had to choose DAST.
Izar Tarandach (14:56):
No, no, I, I, I went there.
Chris Romeo (14:59):
But listen, as an executive, I don't know what
DAST is.
If I'm a COO, for example, Iprobably don't know what DAST
is.
I don't care what DAST is.
Rights.
I care about security, findings.
I care about, um, improvementsthat you're making.
I care about the metrics that Ican go to the board with and
say, when the board looks at meand says, Hey, COO is, is
cybersecurity getting better?
(15:21):
The same, staying the same, oris it getting worse?
I need to be able to make the, Ineed to be able to say, oh,
we're getting better and here'swhy,
Izar Tarandach (15:27):
Chris, that, that's the thing again.
And, and, uh, I'm, I'm guessingthat I'm going to throw like the
unpopular opinion of today.
Those metrics, the, the metricsthat people tend to use today,
and, and I I, I got surprisedby, by what you said back in
Jim's, uh, uh, episode, theydon't really mean anything.
The number of vulnerabilitiesthat you're going to, the number
(15:49):
of weaknesses that you're goingto identify at the end of threat
modeling, uh, session don't meananything.
Chris Romeo (15:55):
Mm-hmm.
Izar Tarandach (15:56):
Because there are so many different factors
impacting that.
Chris Romeo (16:02):
Okay.
So let's take...
Izar Tarandach (16:03):
What's meaningful is what's the
coverage of the, the system thatyou are threat modeling.
What's meaningful is what arethe threats that you are
evaluating?
What's meaningful is...
Chris Romeo (16:11):
What's the mitigations?
Izar Tarandach (16:13):
The mitigations, if any.
What are the developers doingwith what they're learning?
Are they learning something?
Are you accepting risk?
And why are you accepting thatrisk?
Those are all intangibles thatyou can explain.
You can tell a story of risk toan executive.
Chris Romeo (16:30):
Executives definitely speak risk, 100
percent.
Izar Tarandach (16:32):
But you are not, but that risk is not going to
come in the in, in the, in thelanguage of numbers and three
callers.
Chris Romeo (16:40):
Oh, it has to.
'cause I don't speak anythingelse as an executive.
You can't, but you can't explainto me why something is a high
critical finding, because I'mnot gonna understand the
technical...
Some I'm, I'm being, I'm beingunfair here, I'm being
stereotypical.
Let me say, most executives arenot gonna follow you on a
journey of why something is sucha big problem that you found and
mitigated.
Izar Tarandach (17:00):
Look, I, I can use a laser printer to print the
most high-def report ever.
Or I can go in front of theboard with a set of crayons and
explain to them.
In the language that they needto understand, not that they
want or that they can't, thatthey need to understand because
they're dealing with a thousandother very, very complicated and
important factors.
Chris Romeo (17:21):
Mm-hmm.
Izar Tarandach (17:22):
I can use my three crayons to say this is
where we are today in terms ofrisk.
Inot everything needs to come interms of numbers.
Chris Romeo (17:29):
Yeah.
But that's how I, to how, how,how do you, how do I generate a
report?
I have to, I have to, you know,there's this thing called
Sarbanes Oxley, which I hate thefact that I know what this is,
but I have to, I have to, as anexecutive.
I have to sign a document andsend it to the United States
government that says that you,how you explain the risk to me
(17:50):
is something that we actuallyare, we're...
I'm, I'm putting my freedom andmy livelihood and all of my
money on the line.
So I'm not gonna let you come inthere and draw me a crayon
picture.
'Cause I'm gonna say Izar, areyou gonna write me a check if I,
if, if they come to put me injail and take all my money away
and sue me for lying on thisbecause of your, crayon picture?
Izar Tarandach (18:12):
But now we are talking two different
approaches.
We, we have the risk andgovernance people doing the
amazing work that they do.
That's, Lord knows if Iunderstand that, that they can
put those things in terms thatSOCs understands and SOCs
receives and, and accepts, andme as an, uh, uh, I won't even
say as an AppSec person, but as,as a security person to come and
say, listen, this is where oursecurity posture is today.
(18:35):
This is where it was, yeah.
A month ago, and these are thethings that I'm going to do in
this month so that we...
Chris Romeo (18:41):
mm-hmm.
Izar Tarandach (18:41):
We're better next month and this is how I
translate risk.
Okay.
Chris Romeo (18:45):
Now we've morphed the conversation though.
We went from Matt making aninvestment, conver having an
investment conversation with meabout rolling out threat
modeling and moving it out intoan organization where I'm gonna
write a check from my budget toa risk and compliance.
Izar Tarandach (19:00):
No, no, no.
We we're still at the sameplace.
We're still at the same place.
Chris Romeo (19:03):
There's two different things,'cause if Matt
wants me to invest, that's adifferent conversation than what
am I gonna sign on the SarbanesOxley report.
Izar Tarandach (19:10):
Look, we're in this, I, I think that we are
still in the same place becauseright now what we are working on
is on the pitch.
That we're going to come to theboard and say, I need time and I
need money to do threatmodeling.
Chris Romeo (19:21):
Okay.
Izar Tarandach (19:21):
'Cause I'll be able to express that risk in
these ways that are not theSarbanes Oxley numbers, but that
are going to give you anunderstanding of what's the risk
and the residual risk and thethings that we are doing to
lower that risk across theorganization.
Just because we are going totake the time to sit back and
think what could go wrong.
Chris Romeo (19:40):
Okay.
Izar Tarandach (19:40):
Right.
Chris Romeo (19:41):
Alright, I'll, I'll play along.
What, what are you gonna giveme?
Izar Tarandach (19:45):
I'm gonna give look at visibility.
Chris Romeo (19:47):
That I can understand.
No, visibility is a, is adescriptor.
What, what am I, what, what am Igonna have visibility of?
What's the subject that I'mgoing to gain?
Izar Tarandach (19:58):
Your actual attack surface and what you're
doing about it.
Chris Romeo (20:01):
Uh, what is an attack surface?
Izar Tarandach (20:04):
How exposed you are.
Chris Romeo (20:08):
Exposed to what?
Izar Tarandach (20:09):
Everything.
Chris Romeo (20:09):
I'm playing executive here.
Everything?
Izar Tarandach (20:11):
Anybody who comes and tries to take a bite
out of you, we are going to puttogether threat modeling, threat
intelligence, threat everything.
Chris Romeo (20:18):
So you're gonna send me a pile of threat models
and threat intelligence reports?
Izar Tarandach (20:23):
No.
I'm going to tell you a story.
Chris Romeo (20:25):
Okay, so what are you gonna, what's the tell
story?
What, what's gonna be the...
I'm kind of walking into acorner here.
What's gonna be the, what'sgonna be the backbone of that
story that you tell me?
Izar Tarandach (20:33):
Backbone of that story?
Chris Romeo (20:34):
It's gonna be anecdotes.
'cause I can't, I can't go tocourt with anecdotes.
No.
Our lawyer, our legal team willnot support anecdotes in court.
Izar Tarandach (20:40):
It's, it's observation, observational.
It's, okay, we know that this isthe things that we are
defending.
Chris Romeo (20:46):
Okay.
Izar Tarandach (20:46):
We know that these are the things that are
trying to attack it because ofA, B, C, D, E.
Chris Romeo (20:51):
How will I know...,
Izar Tarandach (20:52):
These are the reasons why they would.
Chris Romeo (20:54):
How will I measure what the most important things
are so that I can explain it toour legal counsel?
Izar Tarandach (21:04):
So
Chris Romeo (21:07):
I know the answer to my question, but I just want
you to say it.
Izar Tarandach (21:09):
No, no.
The, the, the dance, the dancehere goes from risk...ification
to, to, to, uh, toprioritization, right?
And over time I came tounderstand it to me at least
personally.
Those are two different things.
Chris Romeo (21:25):
Yeah.
Izar Tarandach (21:25):
One thing is to say how much risk you run there
and the other say how muchyou're going to prioritize
whatever fix needs to comefirst.
Chris Romeo (21:30):
But my point is, you're gonna have to give me
data.
You're gonna have to gimmemetrics.
Izar Tarandach (21:34):
But my point is that that data doesn't come in
numbers always.
That data can be expressed indifferent ways.
Chris Romeo (21:40):
Okay, I see where you're going.
Izar Tarandach (21:41):
It still tells us the same story.
Chris Romeo (21:42):
You can gimme yes, you can gimme a red, yellow,
green.
That's a, that's a fine thingthat happens in, in these
conversations all the time.
Izar Tarandach (21:48):
But not only that, I can tell you where we
were last month.
So I can give you a trend.
Chris Romeo (21:52):
Yeah, I want trend, but I really want trends that
are, as an executive, I don'twanna know.
Red, yellow, green, right?
I want some more data because Iam technically savvy.
I do understand how things workand I want to see a trend line.
I wanna see if how we're gettingbetter.
So I want you to gimme a scorefor on a per product or
application basis for lastquarter and this quarter.
(22:15):
And I wanna look at thosenumbers and I wanna see those
numbers trending up.
'cause if I see those numberstrending down, we've got
problems.
Izar Tarandach (22:22):
So it it, it's the difference between intel...
giving somebody an intelligenceanalysis and giving them the raw
intelligence.
Okay.
You, you're leaving them to dotheir own analysis.
If I give you a bunch ofindicators, numbers that you
decided, because me as a, as asecurity professional with
experience, I may have decidedas, as I have done that, many of
these numbers actually don't sayanything.
(22:45):
They're just numbers.
Okay.
You can go and, and, and buildyour story.
And perhaps your story isdifferent from mine.
Chris Romeo (22:51):
Mm-hmm.
Izar Tarandach (22:52):
Because mine comes with an interpretation.
Comes with an analysis.
Chris Romeo (22:56):
Mm-hmm.
Izar Tarandach (22:56):
Coming, comes with an understanding of what's
happening out there in thebusiness.
Chris Romeo (22:59):
And over time, I'm gonna come to trust your
analysis more.
The first time you deliver it tome, I'm not gonna trust your
analysis very much.
'cause once again, I'm the onewhose butt's on the line.
Izar Tarandach (23:10):
Yep.
Chris Romeo (23:10):
If, if, if what you told me is not correct, they're
coming for me.
They're not coming for you.
I might try to come for youafter that, but for me, because
I'm the one who wrote thesignature on the line of the
reports that went to the federalgovernment and got filed with
the stock exchange and all ofthose things, right?
And so over time, I'm gonna cometo trust you more as a, like, if
(23:31):
you're my CISO for example,you're gonna, I'm gonna, you're,
I'm gonna start building tru...
My level trust level's gonna goup over time too.
I'm gonna get to the point whereI'm like, whatever Izar tells me
is gold because I trust him.
And I've had, I've looked atsome of the data enough to know,
How he's drawing hisconclusions.
Executives are smarter than I'mgiving them credit for here.
Right?
Like they can look at the rawdata and they didn't get to be
(23:54):
an executive because all theycan do is summarize and...
Izar Tarandach (23:57):
Look when you go to a new doctor, okay, you, you
have a choice.
You can decide to implicitlytrust them because they are a
doctor, or you can say he betterprove himself to be first or
herself or themselves.
Mm-hmm.
Okay.
It's the same thing.
You, you go to any kind ofexpert, you either implicitly
(24:17):
accept the authority or you say,this person has to prove
themselves to me.
Chris Romeo (24:21):
I mean, everybody...
there, there's always one doctorwho graduated at the bottom of
their class.
Izar Tarandach (24:26):
Yep.
Chris Romeo (24:27):
Don't forget that when
Izar Tarandach (24:27):
it doesn't mean, doesn't mean that he's, doesn't
mean that he is, uh, uh, uh,less of a doctor because there
are billions of people whodidn't go to to that class at
all.
Somebody has to be the last.
Matt Coles (24:38):
So let's just be careful here.
We're not talking apples toapples comparison, right?
A doctor is like a consultantversus an employee.
Like the board has anengineering team that hired
somebody to be a, an experthere, right?
They're not asking a thirdparty.
They're asking,
Izar Tarandach (24:57):
no, let, let's go with another one.
You have a lawyer on retainer,you're paying the retainer, but
the first time that you're usingthem, you, you have to make a, a
qualitative decision, are yougoing to trust them as is or are
they going to have to provethemselves?
Chris Romeo (25:10):
I mean, every time I work, even as a small business
owner, every time I, I, I, Idon't just implicitly accept
what my lawyer says.
I think about it for a secondand go, okay, yeah, okay.
I can follow that logic.
I don't just say, because you'rea lawyer, I'm gonna do exactly
what you told me to do,
Izar Tarandach (25:27):
but you're not going to him and saying, give me
a list of the precedent so thatI can go case over case and
decide if your line of, uh,reasoning is the right one or
not.
Chris Romeo (25:36):
That is true as well.
Izar Tarandach (25:38):
You, you, you, you do an informed decision.
There's a difference between aninformed decision and what's the
name of the thing?
Uh, back, uh, backseat, uh,driving.
Uh,
Chris Romeo (25:48):
Yeah.
Izar Tarandach (25:49):
There's a difference between those two
things.
Chris Romeo (25:50):
I mean, so first of all, if I'm a big company
executive, I have the, there's,we have our own legal team, and
those lawyers are technically onthe hook just like I am to some
degree, right?
They're carrying some liabilitybased on the things that they're
telling me.
So it's, it's not quite as, aseasy as it's an outside counsel
(26:11):
and, and there.
Matt Coles (26:13):
But likewise, so is your, so is your CISO or your VP
of engineering who arecommunicating...
Chris Romeo (26:19):
mm-hmm.
Matt Coles (26:20):
...around, let's bring it back to threat
modeling, right?
If I'm doing threat modeling forcybersecurity and, and or want
to do that and deliveringinformation as a CISO to the
board.
The CISO is in the same boat asthe board if something, if
things go south.
Chris Romeo (26:35):
Yeah.
Matt Coles (26:35):
Right.
Chris Romeo (26:35):
I mean, we saw with Uber, right?
Right.
We saw the CISO get brought upon charges.
Now that was a little, I'm notgonna comment.
Matt Coles (26:43):
That was, that was extreme.
Chris Romeo (26:44):
Read the news stories.
There was a little more movingparts to that as to who said
what and who did what andwhatnot.
Right.
Izar Tarandach (26:51):
Right.
But people constantly claim, Idid it to the best of my
abilities.
You can't expect more than thatfrom me.
Matt Coles (26:56):
That's the due diligence part, right?
Izar Tarandach (26:57):
I did.
I did it as well as I could.
What I did was reasonable.
Right?
Yeah.
Now, to bring that back to theROI of threat modeling, if we
consider that this begins atthe, just because there, there
should be, there must be anhierarchy.
I'm going to call it like that,the lowest levels of the rung.
Okay.
Then it floats up and it floatsup and it floats up and it
(27:18):
brings that, that picture ofyour attack surface, the risk
you under the, the residualrisk, all that good stuff over
time.
That picture is bound to notonly if you do everything right,
to not only get more clear andmore visible, which is not
always the case, and we knowthat very well.
(27:39):
But it's going to improve aswell, because as we have said, X
number of times, threat modelingis evolutionary.
People start sucking at it.
I sucked at it.
I like to think that I gotbetter at it over time.
Right?
Yeah.
Matt Coles (27:54):
Yeah.
Izar Tarandach (27:54):
So the important thing here is that the, the
return of investment here is, isagain, And analog to the, the,
the, the training saying like,what happens if we train them
and they leave?
What happens if we don't trainthem and they stay?
Chris Romeo (28:11):
Mm-hmm.
Izar Tarandach (28:12):
So what happens if we threat model and we figure
everything out?
What happens if we don't threatmodel and somebody else is going
to tell us what we forgot?
Matt Coles (28:20):
Somebody else will figure it out.
Chris Romeo (28:22):
Somebody else will threat model for us.
Izar Tarandach (28:23):
Yeah.
Chris Romeo (28:24):
Alright, so we don't, we only have a few
minutes left.
Lemme take my executive hat off.
Matt Coles (28:28):
Oh, I actually had one other thing for the
executive.
Chris Romeo (28:30):
Oh, oh, hold on, I'll put my executive hat.
Hold on.
I can just pick it up.
Izar Tarandach (28:32):
He, he wants a raise.
He wants a raise.
Chris Romeo (28:34):
Alright, Matt, I'm back as executive.
Matt Coles (28:35):
So benefit and, and the last benefit, and I'm gonna
just drop it out there, quickcomments if you want.
It's not revenue protection, butit's definitely revenue
generating at some point, threatmodeling along with other
security activities in thelifecycle will be a barrier to
sales.
(28:56):
Right?
We already see this with the,with the CISA attestation for
the federal government, ifyou're selling to the federal
government or your criticalclinical infrastructure, you
have to develop the attestationform, which means you've done
some amount of security.
Now, the threat modeling isactually is as we know, not part
of that directly, but at somepoint that likely is, is likely
(29:17):
to to be introduced.
Chris Romeo (29:19):
Mm-hmm.
Matt Coles (29:19):
In which case not doing it isn't just revenue
protection, meaning reducing myrisk.
But now it is directly enablingsales to occur because I now
have met the criteria forprocurement.
Chris Romeo (29:32):
I mean,
Izar Tarandach (29:33):
and my last line, my last line.
Of all the activities in theSDLC threat modeling is the one
that improves all the others.
I, I, I said that many times,and I'll say it again.
You can use, use it as a hangerto put all the other activities,
(29:53):
hang it on, on, on it, and theywill be better.
If you have a good threat model,your security testing is going
to be better.
Chris Romeo (29:58):
Mm-hmm.
Izar Tarandach (29:59):
If you have good threat modeling, your, your
secure implementation is goingto be better.
Matt Coles (30:03):
And your vulnerability response will be
better.
Your, RCA exercise will bebetter.
Chris Romeo (30:07):
Yeah.
Izar Tarandach (30:08):
As a return of investment, you are multiplying
the efficiency of all the otherthings that you do, including
best.
Chris Romeo (30:15):
You gotta, you gotta prove that to me though,
like, that's my point though, isyou need, you gotta gimme data.
You can't just tell me it'simproving it.
Like you can't come to a meetingwith an executive and say, well,
threat modeling is improving allthese things.
And then just stop.
Because they're gonna say, okay,how, let's, how is it improving?
Gimme some data.
Let me, let me see how youmeas..., how you drew that
(30:35):
conclusion.
Izar Tarandach (30:36):
Let's get two teams and threat model on one
and not threat model on theother.
And use the artifacts of thethreat model the right way on
one and not do it on the other.
Chris Romeo (30:45):
Yeah.
Izar Tarandach (30:45):
And then let's compare the overall happiness of
the developers.
Chris Romeo (30:48):
I mean, I think, I mean, listen, I'm an executive.
I don't care about the happinessof developers.
I care about how much...
Izar Tarandach (30:53):
As much as it hurts, I hear you.
Chris Romeo (30:56):
I know I'm being, this is the raw version.
This isn't me as the, youknow...
.This is what, no, when, when, Iwouldn't really say that.
Izar Tarandach (31:01):
But when I say the happiness of the employees,
and, and this is my closingstatement.
I go back to what I have thoughtto myself is the right way of
going about the return ofinvestment of threat modeling.
It's asking the people who areinvolved in the process, would
you do it again?
But that's, that, that's way,that's way under the level of
(31:21):
the executive hat that you arewearing.
Chris Romeo (31:23):
I mean?
That's,
Izar Tarandach (31:24):
that's the people who are actually doing
the thing.
Chris Romeo (31:26):
If you collected that data for me though, and you
showed me, Hey, with our pilotgroup, we did a, we had a
business unit, we had everybodythreat model for a quarter.
And the funny thing is, with anNPS style survey, we, we
averaged 8.975.
Meaning I don't, people werepromoters of this.
Or you could just say, well, youcould do a binary, well, you do
(31:47):
it again.
Um,
Izar Tarandach (31:49):
I don't survey,
Chris Romeo (31:50):
but the, but, but if you did though, you could
then show me.
Now, isn't it interesting thatalmost a hundred per or 90% of
people that did this threatmodeling process said they would
do it again?
Because they see the value.
Now we're talking about, nowwe've got data.
Now I can go, well, Matt, maybewith your little rollup, maybe
we should do three businessunits.
Let's roll this thing up tothree business units now.
(32:11):
Yeah.
And and that's data that wouldstand behind.
Yeah.
See how, and, and I can, andthen when the board looks at me
and says, Mr.
COO, why did you raise, why didyour, you need to increase your
budget by this amount?
Well, because we did a pilotwith this thing.
We had really good results.
We have some data to back it up.
We think it's really gonna dothe things Matt was talking
about here.
(32:31):
It's gonna introduceopportunities for new revenue.
It's gonna protect old revenue.
Um, we made this investment.
Here's what we saw from thedata.
So now we're gonna invest inthree.
We're gonna roll this out tothree business units.
You know what?
The board may come back and say,you know what?
Why don't you do that with five?
Let's increase the budget alittle bit because we like the,
we like the trend line of this.
It's improving our cybersecuritystory.
(32:53):
So there, there's where dataenables you to kind of make,
make things work.
And since we're all inagreement, that'll be the end of
the security table for thisweek.
Thanks Izar.
Thanks Matt.
That was a great, uh, dialogue.
I will take my executive hatoff.
Set it on the table over here soI can back, go back to being
normal.
Uh, just kidding executives outthere.
I'm not.
I'm not, I'm just, I'm justpoking fun and, and, uh, I was
(33:15):
being somewhat stereotypical inthe average, uh, executive.
I understand lots of executivesare different levels of
technical knowledge andeverything else.
And so, um, don't take offense.
It wasn't intended.
I'm just, we were trying to, wewere trying to reflect a, a
conversation on, on how thingswould actually be thought...
Izar Tarandach (33:31):
Spoken like a true executive.
Chris Romeo (33:36):
All right.
Thanks everybody.
Thanks for joining this episode.
Hey folks.
Welcome to another episode ofthe Security Table.
This is Chris Romeo, joined bymy good friends, Izar Tarandach
and Jerry Garcia.
No, no, no.
Sorry.
Sorry.
I forgot I was a little, little.
A little confused for a second,but it's actually Matt Cole, not
Jerry,
Izar Tarandach (00:28):
just because he looks a bit, a bit hazy.
Chris Romeo (00:32):
Just look, maybe he could sing a couple of, uh, a
couple of bars for us, maybe?
Matt Coles (00:37):
Absolutely not.
Chris Romeo (00:38):
Absolutely not.
Thought we might get a, youknow, truckers or, you know,
lemme see.
Something Magnolia Sweet Magno.
I don't remember.
Matt Coles (00:48):
I have no idea what what you're talking about.
Chris Romeo (00:50):
Grateful Dead songs, but, so yes.
Matt Coles (00:55):
College was a haze because of, uh, great Grateful
dead song.
Chris Romeo (00:59):
There you go.
You're not, you're not oldenough for that to line up,
unfortunately.
Matt Coles (01:04):
Tell that to the rest of my floor in college.
Chris Romeo (01:07):
So we're gonna continue our.
Jaunt into the world of threatmodeling.
So in our previous episode, wehad a great conversation with
Jim Manco about where he sits inthe world of threat modeling,
and we heard about how he, uh,where, where his ideas were kind
of coming from.
I thought that was prettyeye-opening.
His view of consultants and howthey abused threat modeling in
(01:28):
the early years of AppSec, Ithought was, uh, was a good
perspective.
It was one that I hadn't.
Necessarily seen firsthand beinginside a product company and,
and not having consultants thatwere being, that were doing the
threat modeling for us.
I thought that was a reallyinteresting viewpoint, but we
want to talk about the return oninvestment for threat modeling.
(01:49):
And so I thought we'd start by,let's define return on
investment just for those peoplethat maybe, um, aren't as deep
into the business side of.
Running a company or being apart of a big company where
return on investment's a bigdeal.
Matt, why don't you kick thatone off for us?
What is return on investment?
'cause I know Izar likes to justcome in and say same or
(02:12):
basically flip around what yousaid completely.
So, Matt, return on investment.
What is it?
Matt Coles (02:18):
Um, so actually, uh, I'm gonna defer back to you
because I think you've got the,the, the broader business sense
here.
Uh, so I.
I mean, I know generally whatROI is, right?
It's a common term we usethroughout the security world
and, and obviously, uh, if youhave any of the big name
certifications, ROI must havecome up in those.
But in a, in a bare sense, it's,it's a cost benefit system,
(02:44):
right?
It's, it's a system of how muchmoney do you spend versus what
do you get in return.
There's more to that.
So I wanna throw it back to youactually,'cause I think you have
a good sense of this.
Obviously, you're verysuccessful in, uh, in running a
business.
So you've done ROI and you'veeducated others on how to do ROI
for security.
So let me throw it back to you.
(03:05):
Help us understand what ROI isis here in context.
Chris Romeo (03:08):
Yeah.
I think your, your definition isright on as far as it is, what's
the value that's being generatedfor the dollars for the
resources could be humanresources as well that are being
expended towards a given cause.
And so when I think ROI, I thinkthere's kind of startup ROI
world, and then there's bigcorporate ROI world.
(03:29):
And in the big corporate ROI.
Often executives are looking forsome type of metric of what am I
getting for the investment thatI'm making of dollars in people,
security team.
Executives don't care about howmany alerts are generated, how
(03:50):
many SAST findings there were,how many, uh, vulnerable third
party packages we have.
Executives don't think likethat.
Executives run businesses andreturn on investment is as
simple as saying, you're askingme Izar to spend$500,000 and
expend 500 people each a week oftheir time.
What am I getting?
(04:11):
What are you providing me as theresult of me making this
investment?
And so it's really a businesscase conversation.
It's, it's, are you providing meenough value as an executive
where I can say, okay, I'llinvest that money because I see
what I'm gonna get out of it.
That's really return oninvestment in, in the startup
world, it's, it's a lot smaller'cause there's not as many
(04:31):
people talking about it, but itis still important.
Like it, when I was runningSecurity Journey for example, we
would be often have a decisionin front of us and it would have
a dollar cost and we'd have to,we'd have to wrestle with as a
small group of executives tosay, does that make sense?
Should we move forward with thisor do we hold off and wait
another quarter?
(04:51):
Or, you know, before we investthese dollars.
So that's kind of my take onreturn on investment.
Hopefully it, it helped Matt to,to provide some context from
where I'm coming from.
Matt Coles (04:59):
Oh, absolutely.
Um, so if I could just, Put somethoughts in here and then is
our, I mean, we're good.
We'll just unleash the, unleashthe Kraken Izar uh, so, uh, you
know, just, you just keepdrinking your monster over there
and get, get pumped up for this.
Uh, so if we think about costand cost and benefit, right?
(05:19):
How much we spend, obviously,you know, it's, it's people and
time and te and tools andtechnology to not only do threat
modeling, but also to manage theresults.
Right.
So how many people do you need?
Do you, do you need bring inconsultants?
Are you doing training ahead oftime or does everyone know what
they're doing?
And then can you actuallyexecute?
(05:41):
But I would suggest at least theinitial benefit of threat
modeling is actually in terms ofrevenue protection and, and
preventing rework because theoutcomes of threat modeling are,
uh, issues that you find thatmay influence the design that
(06:01):
need to be fixed.
You fix those upfront, thatimproves your design, but really
what you're doing is preventingvulnerabilities and, and other
things that will appear later inthe development cycle and things
that will get released tocustomers resulting in
effectively revenue protectionas your benefit.
Chris Romeo (06:20):
Can I put my, my executive hat on for a second?
Absolutely, please do.
And we can have a little, we canhave a little, uh, mock kind of
approach to how this mightactually go down in a business.
Because when I hear what,basically what you just said, my
immediate question to you thenis, how are you gonna prove that
to me?
How are you gonna prove thatthere's less rework?
(06:41):
What's your metric gonna bethat's gonna drive this to allow
me to say, okay, now Matt toldme this and this and that, and
things are gonna be better if wemake this investment.
How am I gonna measure you?
What, what number are you gonnaprovide for me that lets me
measure whether you were tellingme the truth when you set this
business case up for?
Matt Coles (06:58):
That's a great question.
I don't have necessarily have agood answer for you.
Izar?
Izar Tarandach (07:06):
Well, in the past five minutes, I have seen
the past 20 years of my life runin front of my, my eyes.
Right?
Matt Coles (07:14):
Are we that old?
my God.
Yeah.
Izar Tarandach (07:17):
Amazing.
Least.
So yeah, that time passes whenyou're having fun uhhuh.
But, uh, you know, the, the, thething is the the, when I was a
young threat modeling padawanand somebody asked me, why
should I do this thing?
The question itself wasunthinkable to me.
Well, what do you mean what,what do you get if you do this?
(07:41):
Don't you realize that if youdon't do this, the whole world
is going going to come crumblingdown and the space time
continuum is going to fold onitself and there's going to be
one big black hole where yourcompany was?
Turns out that I was wrong andthat there are people who will
gladly go about their liveswithout doing that.
(08:04):
And, uh, that conversation withJim, for me, it was enlightening
on, on, on a lot of differentfields, but his insistence on
the wasteful side of threatmodeling showed me a, a couple
of things.
First, that unfortunately Jimneeds to get better informed
(08:25):
about the new ways of doingthreat modeling that have
flourished in the past fewyears.
So where we, we, we haveaddressed a lot of that waste.
But not only that, not not onlythe waste we have, we have
addressed in general.
The effort and what to take outof the, of the, uh, the process.
(08:47):
And, and I think that today it'seasy to say it, it's actually
not easy.
It's, it's clear to say thatthreat modeling has evolved to a
point where we are not onlyidentifying and mitigating and
accepting risk through designweaknesses, we are going way
beyond in terms of understandingwhat are the systems that we are
(09:08):
building, which are forevergetting more and more and more
complicated.
We, we, we are giving, uh, uh,transparency over these things
that we are putting out thereand that are responsible for so
much of, sorry, whatcivilization is doing right?
And we have extended intoprivacy.
We have extended into so manydifferent fields that deal with
(09:28):
this whole experience that welive today.
That the question to me, has tobe turned around and say, how
much are you going to lose ifyou don't?
Especially with us coming to apoint where people are starting
to talk about, again, that wordliability.
When you, when you write, uh, asoftware and you put it out and
(09:50):
the, the, the flag that thethree of us have been bending
about of what's reasonablesecurity, is there anything more
reasonable in terms of securitythan taking time to sit back and
think what could possibly gowrong?
If you have to to discuss, isthis something that I want to do
or not?
I would say that you have asignificantly more deep problem
(10:14):
than what's the ROI on thisthing.
Chris Romeo (10:16):
Yeah, but remember I'm an executive, okay?
And if you tell me that I needto somehow extract the value
myself out of the beauty ofthreat modeling, I'm not gonna
do that.
Because I don't think, I'm not asecurity person.
I'm a business person.
I care about dollars and centsand bottom line, I have an, I
have a number I gotta hit thisquarter to make Wall Street
(10:38):
happy.
Yeah.
So let me, let me add, that'swhat I care about.
So that's, so that's why yougotta tell, you can't, you can't
flip it back on me.
I'm not gonna do it.
I'm just, no way...
Matt Coles (10:46):
can I just add
Chris Romeo (10:46):
that's happening.
Matt Coles (10:47):
I just have one go.
Izar Tarandach (10:49):
Just before we go there, I, I have an
unflinching admiration forpeople who are able to get a pot
of water, throw all the data oftheir business inside, let it
simmer for a while, and come outof something that they call a
threat modeling a threat model.
That's, that's great, especiallyif, uh, tomato paste on it.
(11:11):
But, uh, uh, the, the, the, thepoint is that not everything
translates down to dollars.
At some point we have to moveaway from those numbers and say
it's, it's, it's, it's a qualitything.
It's not a quantity thing.
Chris Romeo (11:24):
Everything boils down to dollars in business, the
executive level.
No, no.
I'm still got my executive haton.
Izar Tarandach (11:31):
Yes.
Matt Coles (11:31):
So let, let, let me introduce it.
Chris Romeo (11:34):
Let Matt, let Matt get in here.
Matt Coles (11:36):
I'm gonna, so I'm gonna, I wanna add something to
the benefit column here and, andI think this will maybe talk a
little bit closer to what you,with your executive hat on is
thinking.
So revenue protection is alittle bit more, is a little
amorphous.
We can talk about things likereduced support calls.
Better or improved customercustomer engagement, improved
(11:57):
customer acceptance.
But Izar, you brought upsomething very particular.
So we've seen the past coupleyears with the, uh, US uh,
executive order aroundcybersecurity and the CISA,
secure by Design, secure bydefault, uh, and the, the SSDF,
uh, you know, secure softwaredevelopment framework from NIST
and, and, and the collection ofagencies across the world that
(12:18):
are adopting those similarpractices.
If we look at threat modeling asa, an aspect of due diligence,
and now I'm gonna bring in thelegal aspect here, due
diligence, the thing that you doto ensure that your software is
free of, of easily discoverable,discoverable vulnerabilities,
and so that you can meetregulatory compliance and legal
(12:41):
expectations and obligations.
Threat modeling becomes anaspect of due diligence.
Due diligence drives regulatorycompliance.
Regulatory compliance means thatyou have revenue protection, and
now you have a dollar and centsdiscussion.
Does that, does that meet yoursmell test there, Mr.
Executive?
Chris Romeo (12:58):
I mean, I think you, I think you're, I think
you're taking me on a, apathway.
I'm, I'm following you on thepathway, but I still need more.
It's still too nebulous to sayrevenue protection and, um, due
diligence and all of thesethings.
They mean stuff, but there's noway I can measure it right?
Now, where, where you were goingwith reduced support calls,
(13:18):
reduced support costs.
Now this is a metric I can lookat.
Because I'm not, we're not gonnaroll threat modeling out in one
week.
We're not gonna say, next weekis threat modeling week.
Get ready everybody.
Everybody's doing nothing else.
We're not doing anything.
We're not building any newfeatures in our company.
All we're doing is threatmodeling, right?
You're gonna roll this out overa period of time, 2, 3, 4
(13:39):
quarters.
If we're talking largeenterprise now, what if we pilot
that as an executive?
I'm gonna say, okay, Matt, Ilove this idea.
I like where you're going here.
Lower support costs and, and,and, and less rework.
Let's do a pilot for a quarterwith a particular business unit
and collect your data and thencome back to me and show me that
(14:02):
you in fact have lower supportcosts and you have less rework
and you, you know, all thosethings you can measure because
I'm not gonna write you a checkfor$5 million on day one.
I'll write you a check for, for$500,000 on day one.
If I believe in the idea and Ithink you got something, but I
can't roll it out enterprisewide without a, without a proof
(14:22):
of concept that shows me yourdata backs up what you were
telling me and Izar's about tofly out his chair.
Izar Tarandach (14:28):
No, but, but, but, but Chris, think about it.
Okay?
What you're telling me is you,Mr.
Executive, are willing to writea check for DAST because DAST
gives you numbers at the end.
Threat modeling doesn't.
Chris Romeo (14:48):
Can I take my hat off for a second and just yell
into the microphone?
No, I mean, I'm, I'm gonna.
Matt Coles (14:53):
You could have chosen of all, any other letter,
but you had to choose DAST.
Izar Tarandach (14:56):
No, no, I, I, I went there.
Chris Romeo (14:59):
But listen, as an executive, I don't know what
DAST is.
If I'm a COO, for example, Iprobably don't know what DAST
is.
I don't care what DAST is.
Rights.
I care about security, findings.
I care about, um, improvementsthat you're making.
I care about the metrics that Ican go to the board with and
say, when the board looks at meand says, Hey, COO is, is
cybersecurity getting better?
(15:21):
The same, staying the same, oris it getting worse?
I need to be able to make the, Ineed to be able to say, oh,
we're getting better and here'swhy,
Izar Tarandach (15:27):
Chris, that, that's the thing again.
And, and, uh, I'm, I'm guessingthat I'm going to throw like the
unpopular opinion of today.
Those metrics, the, the metricsthat people tend to use today,
and, and I I, I got surprisedby, by what you said back in
Jim's, uh, uh, episode, theydon't really mean anything.
The number of vulnerabilitiesthat you're going to, the number
(15:49):
of weaknesses that you're goingto identify at the end of threat
modeling, uh, session don't meananything.
Chris Romeo (15:55):
Mm-hmm.
Izar Tarandach (15:56):
Because there are so many different factors
impacting that.
Chris Romeo (16:02):
Okay.
So let's take...
Izar Tarandach (16:03):
What's meaningful is what's the
coverage of the, the system thatyou are threat modeling.
What's meaningful is what arethe threats that you are
evaluating?
What's meaningful is...
Chris Romeo (16:11):
What's the mitigations?
Izar Tarandach (16:13):
The mitigations, if any.
What are the developers doingwith what they're learning?
Are they learning something?
Are you accepting risk?
And why are you accepting thatrisk?
Those are all intangibles thatyou can explain.
You can tell a story of risk toan executive.
Chris Romeo (16:30):
Executives definitely speak risk, 100
percent.
Izar Tarandach (16:32):
But you are not, but that risk is not going to
come in the in, in the, in thelanguage of numbers and three
callers.
Chris Romeo (16:40):
Oh, it has to.
'cause I don't speak anythingelse as an executive.
You can't, but you can't explainto me why something is a high
critical finding, because I'mnot gonna understand the
technical...
Some I'm, I'm being, I'm beingunfair here, I'm being
stereotypical.
Let me say, most executives arenot gonna follow you on a
journey of why something is sucha big problem that you found and
mitigated.
Izar Tarandach (17:00):
Look, I, I can use a laser printer to print the
most high-def report ever.
Or I can go in front of theboard with a set of crayons and
explain to them.
In the language that they needto understand, not that they
want or that they can't, thatthey need to understand because
they're dealing with a thousandother very, very complicated and
important factors.
Chris Romeo (17:21):
Mm-hmm.
Izar Tarandach (17:22):
I can use my three crayons to say this is
where we are today in terms ofrisk.
Inot everything needs to come interms of numbers.
Chris Romeo (17:29):
Yeah.
But that's how I, to how, how,how do you, how do I generate a
report?
I have to, I have to, you know,there's this thing called
Sarbanes Oxley, which I hate thefact that I know what this is,
but I have to, I have to, as anexecutive.
I have to sign a document andsend it to the United States
government that says that you,how you explain the risk to me
(17:50):
is something that we actuallyare, we're...
I'm, I'm putting my freedom andmy livelihood and all of my
money on the line.
So I'm not gonna let you come inthere and draw me a crayon
picture.
'Cause I'm gonna say Izar, areyou gonna write me a check if I,
if, if they come to put me injail and take all my money away
and sue me for lying on thisbecause of your, crayon picture?
Izar Tarandach (18:12):
But now we are talking two different
approaches.
We, we have the risk andgovernance people doing the
amazing work that they do.
That's, Lord knows if Iunderstand that, that they can
put those things in terms thatSOCs understands and SOCs
receives and, and accepts, andme as an, uh, uh, I won't even
say as an AppSec person, but as,as a security person to come and
say, listen, this is where oursecurity posture is today.
(18:35):
This is where it was, yeah.
A month ago, and these are thethings that I'm going to do in
this month so that we...
Chris Romeo (18:41):
mm-hmm.
Izar Tarandach (18:41):
We're better next month and this is how I
translate risk.
Okay.
Chris Romeo (18:45):
Now we've morphed the conversation though.
We went from Matt making aninvestment, conver having an
investment conversation with meabout rolling out threat
modeling and moving it out intoan organization where I'm gonna
write a check from my budget toa risk and compliance.
Izar Tarandach (19:00):
No, no, no.
We we're still at the sameplace.
We're still at the same place.
Chris Romeo (19:03):
There's two different things,'cause if Matt
wants me to invest, that's adifferent conversation than what
am I gonna sign on the SarbanesOxley report.
Izar Tarandach (19:10):
Look, we're in this, I, I think that we are
still in the same place becauseright now what we are working on
is on the pitch.
That we're going to come to theboard and say, I need time and I
need money to do threatmodeling.
Chris Romeo (19:21):
Okay.
Izar Tarandach (19:21):
'Cause I'll be able to express that risk in
these ways that are not theSarbanes Oxley numbers, but that
are going to give you anunderstanding of what's the risk
and the residual risk and thethings that we are doing to
lower that risk across theorganization.
Just because we are going totake the time to sit back and
think what could go wrong.
Chris Romeo (19:40):
Okay.
Izar Tarandach (19:40):
Right.
Chris Romeo (19:41):
Alright, I'll, I'll play along.
What, what are you gonna giveme?
Izar Tarandach (19:45):
I'm gonna give look at visibility.
Chris Romeo (19:47):
That I can understand.
No, visibility is a, is adescriptor.
What, what am I, what, what am Igonna have visibility of?
What's the subject that I'mgoing to gain?
Izar Tarandach (19:58):
Your actual attack surface and what you're
doing about it.
Chris Romeo (20:01):
Uh, what is an attack surface?
Izar Tarandach (20:04):
How exposed you are.
Chris Romeo (20:08):
Exposed to what?
Izar Tarandach (20:09):
Everything.
Chris Romeo (20:09):
I'm playing executive here.
Everything?
Izar Tarandach (20:11):
Anybody who comes and tries to take a bite
out of you, we are going to puttogether threat modeling, threat
intelligence, threat everything.
Chris Romeo (20:18):
So you're gonna send me a pile of threat models
and threat intelligence reports?
Izar Tarandach (20:23):
No.
I'm going to tell you a story.
Chris Romeo (20:25):
Okay, so what are you gonna, what's the tell
story?
What, what's gonna be the...
I'm kind of walking into acorner here.
What's gonna be the, what'sgonna be the backbone of that
story that you tell me?
Izar Tarandach (20:33):
Backbone of that story?
Chris Romeo (20:34):
It's gonna be anecdotes.
'cause I can't, I can't go tocourt with anecdotes.
No.
Our lawyer, our legal team willnot support anecdotes in court.
Izar Tarandach (20:40):
It's, it's observation, observational.
It's, okay, we know that this isthe things that we are
defending.
Chris Romeo (20:46):
Okay.
Izar Tarandach (20:46):
We know that these are the things that are
trying to attack it because ofA, B, C, D, E.
Chris Romeo (20:51):
How will I know...,
Izar Tarandach (20:52):
These are the reasons why they would.
Chris Romeo (20:54):
How will I measure what the most important things
are so that I can explain it toour legal counsel?
Izar Tarandach (21:04):
So
Chris Romeo (21:07):
I know the answer to my question, but I just want
you to say it.
Izar Tarandach (21:09):
No, no.
The, the, the dance, the dancehere goes from risk...ification
to, to, to, uh, toprioritization, right?
And over time I came tounderstand it to me at least
personally.
Those are two different things.
Chris Romeo (21:25):
Yeah.
Izar Tarandach (21:25):
One thing is to say how much risk you run there
and the other say how muchyou're going to prioritize
whatever fix needs to comefirst.
Chris Romeo (21:30):
But my point is, you're gonna have to give me
data.
You're gonna have to gimmemetrics.
Izar Tarandach (21:34):
But my point is that that data doesn't come in
numbers always.
That data can be expressed indifferent ways.
Chris Romeo (21:40):
Okay, I see where you're going.
Izar Tarandach (21:41):
It still tells us the same story.
Chris Romeo (21:42):
You can gimme yes, you can gimme a red, yellow,
green.
That's a, that's a fine thingthat happens in, in these
conversations all the time.
Izar Tarandach (21:48):
But not only that, I can tell you where we
were last month.
So I can give you a trend.
Chris Romeo (21:52):
Yeah, I want trend, but I really want trends that
are, as an executive, I don'twanna know.
Red, yellow, green, right?
I want some more data because Iam technically savvy.
I do understand how things workand I want to see a trend line.
I wanna see if how we're gettingbetter.
So I want you to gimme a scorefor on a per product or
application basis for lastquarter and this quarter.
(22:15):
And I wanna look at thosenumbers and I wanna see those
numbers trending up.
'cause if I see those numberstrending down, we've got
problems.
Izar Tarandach (22:22):
So it it, it's the difference between intel...
giving somebody an intelligenceanalysis and giving them the raw
intelligence.
Okay.
You, you're leaving them to dotheir own analysis.
If I give you a bunch ofindicators, numbers that you
decided, because me as a, as asecurity professional with
experience, I may have decidedas, as I have done that, many of
these numbers actually don't sayanything.
(22:45):
They're just numbers.
Okay.
You can go and, and, and buildyour story.
And perhaps your story isdifferent from mine.
Chris Romeo (22:51):
Mm-hmm.
Izar Tarandach (22:52):
Because mine comes with an interpretation.
Comes with an analysis.
Chris Romeo (22:56):
Mm-hmm.
Izar Tarandach (22:56):
Coming, comes with an understanding of what's
happening out there in thebusiness.
Chris Romeo (22:59):
And over time, I'm gonna come to trust your
analysis more.
The first time you deliver it tome, I'm not gonna trust your
analysis very much.
'cause once again, I'm the onewhose butt's on the line.
Izar Tarandach (23:10):
Yep.
Chris Romeo (23:10):
If, if, if what you told me is not correct, they're
coming for me.
They're not coming for you.
I might try to come for youafter that, but for me, because
I'm the one who wrote thesignature on the line of the
reports that went to the federalgovernment and got filed with
the stock exchange and all ofthose things, right?
And so over time, I'm gonna cometo trust you more as a, like, if
(23:31):
you're my CISO for example,you're gonna, I'm gonna, you're,
I'm gonna start building tru...
My level trust level's gonna goup over time too.
I'm gonna get to the point whereI'm like, whatever Izar tells me
is gold because I trust him.
And I've had, I've looked atsome of the data enough to know,
How he's drawing hisconclusions.
Executives are smarter than I'mgiving them credit for here.
Right?
Like they can look at the rawdata and they didn't get to be
(23:54):
an executive because all theycan do is summarize and...
Izar Tarandach (23:57):
Look when you go to a new doctor, okay, you, you
have a choice.
You can decide to implicitlytrust them because they are a
doctor, or you can say he betterprove himself to be first or
herself or themselves.
Mm-hmm.
Okay.
It's the same thing.
You, you go to any kind ofexpert, you either implicitly
(24:17):
accept the authority or you say,this person has to prove
themselves to me.
Chris Romeo (24:21):
I mean, everybody...
there, there's always one doctorwho graduated at the bottom of
their class.
Izar Tarandach (24:26):
Yep.
Chris Romeo (24:27):
Don't forget that when
Izar Tarandach (24:27):
it doesn't mean, doesn't mean that he's, doesn't
mean that he is, uh, uh, uh,less of a doctor because there
are billions of people whodidn't go to to that class at
all.
Somebody has to be the last.
Matt Coles (24:38):
So let's just be careful here.
We're not talking apples toapples comparison, right?
A doctor is like a consultantversus an employee.
Like the board has anengineering team that hired
somebody to be a, an experthere, right?
They're not asking a thirdparty.
They're asking,
Izar Tarandach (24:57):
no, let, let's go with another one.
You have a lawyer on retainer,you're paying the retainer, but
the first time that you're usingthem, you, you have to make a, a
qualitative decision, are yougoing to trust them as is or are
they going to have to provethemselves?
Chris Romeo (25:10):
I mean, every time I work, even as a small business
owner, every time I, I, I, Idon't just implicitly accept
what my lawyer says.
I think about it for a secondand go, okay, yeah, okay.
I can follow that logic.
I don't just say, because you'rea lawyer, I'm gonna do exactly
what you told me to do,
Izar Tarandach (25:27):
but you're not going to him and saying, give me
a list of the precedent so thatI can go case over case and
decide if your line of, uh,reasoning is the right one or
not.
Chris Romeo (25:36):
That is true as well.
Izar Tarandach (25:38):
You, you, you, you do an informed decision.
There's a difference between aninformed decision and what's the
name of the thing?
Uh, back, uh, backseat, uh,driving.
Uh,
Chris Romeo (25:48):
Yeah.
Izar Tarandach (25:49):
There's a difference between those two
things.
Chris Romeo (25:50):
I mean, so first of all, if I'm a big company
executive, I have the, there's,we have our own legal team, and
those lawyers are technically onthe hook just like I am to some
degree, right?
They're carrying some liabilitybased on the things that they're
telling me.
So it's, it's not quite as, aseasy as it's an outside counsel
(26:11):
and, and there.
Matt Coles (26:13):
But likewise, so is your, so is your CISO or your VP
of engineering who arecommunicating...
Chris Romeo (26:19):
mm-hmm.
Matt Coles (26:20):
...around, let's bring it back to threat
modeling, right?
If I'm doing threat modeling forcybersecurity and, and or want
to do that and deliveringinformation as a CISO to the
board.
The CISO is in the same boat asthe board if something, if
things go south.
Chris Romeo (26:35):
Yeah.
Matt Coles (26:35):
Right.
Chris Romeo (26:35):
I mean, we saw with Uber, right?
Right.
We saw the CISO get brought upon charges.
Now that was a little, I'm notgonna comment.
Matt Coles (26:43):
That was, that was extreme.
Chris Romeo (26:44):
Read the news stories.
There was a little more movingparts to that as to who said
what and who did what andwhatnot.
Right.
Izar Tarandach (26:51):
Right.
But people constantly claim, Idid it to the best of my
abilities.
You can't expect more than thatfrom me.
Matt Coles (26:56):
That's the due diligence part, right?
Izar Tarandach (26:57):
I did.
I did it as well as I could.
What I did was reasonable.
Right?
Yeah.
Now, to bring that back to theROI of threat modeling, if we
consider that this begins atthe, just because there, there
should be, there must be anhierarchy.
I'm going to call it like that,the lowest levels of the rung.
Okay.
Then it floats up and it floatsup and it floats up and it
(27:18):
brings that, that picture ofyour attack surface, the risk
you under the, the residualrisk, all that good stuff over
time.
That picture is bound to notonly if you do everything right,
to not only get more clear andmore visible, which is not
always the case, and we knowthat very well.
(27:39):
But it's going to improve aswell, because as we have said, X
number of times, threat modelingis evolutionary.
People start sucking at it.
I sucked at it.
I like to think that I gotbetter at it over time.
Right?
Yeah.
Matt Coles (27:54):
Yeah.
Izar Tarandach (27:54):
So the important thing here is that the, the
return of investment here is, isagain, And analog to the, the,
the, the training saying like,what happens if we train them
and they leave?
What happens if we don't trainthem and they stay?
Chris Romeo (28:11):
Mm-hmm.
Izar Tarandach (28:12):
So what happens if we threat model and we figure
everything out?
What happens if we don't threatmodel and somebody else is going
to tell us what we forgot?
Matt Coles (28:20):
Somebody else will figure it out.
Chris Romeo (28:22):
Somebody else will threat model for us.
Izar Tarandach (28:23):
Yeah.
Chris Romeo (28:24):
Alright, so we don't, we only have a few
minutes left.
Lemme take my executive hat off.
Matt Coles (28:28):
Oh, I actually had one other thing for the
executive.
Chris Romeo (28:30):
Oh, oh, hold on, I'll put my executive hat.
Hold on.
I can just pick it up.
Izar Tarandach (28:32):
He, he wants a raise.
He wants a raise.
Chris Romeo (28:34):
Alright, Matt, I'm back as executive.
Matt Coles (28:35):
So benefit and, and the last benefit, and I'm gonna
just drop it out there, quickcomments if you want.
It's not revenue protection, butit's definitely revenue
generating at some point, threatmodeling along with other
security activities in thelifecycle will be a barrier to
sales.
(28:56):
Right?
We already see this with the,with the CISA attestation for
the federal government, ifyou're selling to the federal
government or your criticalclinical infrastructure, you
have to develop the attestationform, which means you've done
some amount of security.
Now, the threat modeling isactually is as we know, not part
of that directly, but at somepoint that likely is, is likely
(29:17):
to to be introduced.
Chris Romeo (29:19):
Mm-hmm.
Matt Coles (29:19):
In which case not doing it isn't just revenue
protection, meaning reducing myrisk.
But now it is directly enablingsales to occur because I now
have met the criteria forprocurement.
Chris Romeo (29:32):
I mean,
Izar Tarandach (29:33):
and my last line, my last line.
Of all the activities in theSDLC threat modeling is the one
that improves all the others.
I, I, I said that many times,and I'll say it again.
You can use, use it as a hangerto put all the other activities,
(29:53):
hang it on, on, on it, and theywill be better.
If you have a good threat model,your security testing is going
to be better.
Chris Romeo (29:58):
Mm-hmm.
Izar Tarandach (29:59):
If you have good threat modeling, your, your
secure implementation is goingto be better.
Matt Coles (30:03):
And your vulnerability response will be
better.
Your, RCA exercise will bebetter.
Chris Romeo (30:07):
Yeah.
Izar Tarandach (30:08):
As a return of investment, you are multiplying
the efficiency of all the otherthings that you do, including
best.
Chris Romeo (30:15):
You gotta, you gotta prove that to me though,
like, that's my point though, isyou need, you gotta gimme data.
You can't just tell me it'simproving it.
Like you can't come to a meetingwith an executive and say, well,
threat modeling is improving allthese things.
And then just stop.
Because they're gonna say, okay,how, let's, how is it improving?
Gimme some data.
Let me, let me see how youmeas..., how you drew that
(30:35):
conclusion.
Izar Tarandach (30:36):
Let's get two teams and threat model on one
and not threat model on theother.
And use the artifacts of thethreat model the right way on
one and not do it on the other.
Chris Romeo (30:45):
Yeah.
Izar Tarandach (30:45):
And then let's compare the overall happiness of
the developers.
Chris Romeo (30:48):
I mean, I think, I mean, listen, I'm an executive.
I don't care about the happinessof developers.
I care about how much...
Izar Tarandach (30:53):
As much as it hurts, I hear you.
Chris Romeo (30:56):
I know I'm being, this is the raw version.
This isn't me as the, youknow...
.This is what, no, when, when, Iwouldn't really say that.
Izar Tarandach (31:01):
But when I say the happiness of the employees,
and, and this is my closingstatement.
I go back to what I have thoughtto myself is the right way of
going about the return ofinvestment of threat modeling.
It's asking the people who areinvolved in the process, would
you do it again?
But that's, that, that's way,that's way under the level of
(31:21):
the executive hat that you arewearing.
Chris Romeo (31:23):
I mean?
That's,
Izar Tarandach (31:24):
that's the people who are actually doing
the thing.
Chris Romeo (31:26):
If you collected that data for me though, and you
showed me, Hey, with our pilotgroup, we did a, we had a
business unit, we had everybodythreat model for a quarter.
And the funny thing is, with anNPS style survey, we, we
averaged 8.975.
Meaning I don't, people werepromoters of this.
Or you could just say, well, youcould do a binary, well, you do
(31:47):
it again.
Um,
Izar Tarandach (31:49):
I don't survey,
Chris Romeo (31:50):
but the, but, but if you did though, you could
then show me.
Now, isn't it interesting thatalmost a hundred per or 90% of
people that did this threatmodeling process said they would
do it again?
Because they see the value.
Now we're talking about, nowwe've got data.
Now I can go, well, Matt, maybewith your little rollup, maybe
we should do three businessunits.
Let's roll this thing up tothree business units now.
(32:11):
Yeah.
And and that's data that wouldstand behind.
Yeah.
See how, and, and I can, andthen when the board looks at me
and says, Mr.
COO, why did you raise, why didyour, you need to increase your
budget by this amount?
Well, because we did a pilotwith this thing.
We had really good results.
We have some data to back it up.
We think it's really gonna dothe things Matt was talking
about here.
(32:31):
It's gonna introduceopportunities for new revenue.
It's gonna protect old revenue.
Um, we made this investment.
Here's what we saw from thedata.
So now we're gonna invest inthree.
We're gonna roll this out tothree business units.
You know what?
The board may come back and say,you know what?
Why don't you do that with five?
Let's increase the budget alittle bit because we like the,
we like the trend line of this.
It's improving our cybersecuritystory.
(32:53):
So there, there's where dataenables you to kind of make,
make things work.
And since we're all inagreement, that'll be the end of
the security table for thisweek.
Thanks Izar.
Thanks Matt.
That was a great, uh, dialogue.
I will take my executive hatoff.
Set it on the table over here soI can back, go back to being
normal.
Uh, just kidding executives outthere.
I'm not.
I'm not, I'm just, I'm justpoking fun and, and, uh, I was
(33:15):
being somewhat stereotypical inthe average, uh, executive.
I understand lots of executivesare different levels of
technical knowledge andeverything else.
And so, um, don't take offense.
It wasn't intended.
I'm just, we were trying to, wewere trying to reflect a, a
conversation on, on how thingswould actually be thought...
Izar Tarandach (33:31):
Spoken like a true executive.
Chris Romeo (33:36):
All right.
Thanks everybody.
Thanks for joining this episode.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Therapy Gecko
An unlicensed lizard psychologist travels the universe talking to strangers about absolutely nothing. TO CALL THE GECKO: follow me on https://www.twitch.tv/lyleforever to get a notification for when I am taking calls. I am usually live Mondays, Wednesdays, and Fridays but lately a lot of other times too. I am a gecko.