May 19, 2026 • 55 mins
Got a question or comment? Message us here!
When the Canvas LMS went down, thousands of institutions came to a halt, right in the middle of finals. In this episode, we break down what really happened, what data may have been exposed, and why this incident is a wake-up call for every organization relying on SaaS platforms.
From vendor risk and contract blind spots to business continuity failures, we unpack the real lessons security leaders need to hear, and what you should be doing right now to prepare for the next breach.
Watch full episodes at youtube.com/@aliascybersecurity.
Listen on Apple Podcasts, Spotify and anywhere you get your podcasts.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_02 (00:00):
And I had a customer that reached out, one of our
customers reached out to me witha screenshot saying, um, hey, we
had a user navigate to thistraining website that we use,
and there's this shiny hunters,and we all take a screenshot and
send it to us.
And they were super concerned.
And this wasn't even a highered.
Yeah.
This was an organization thatjust utilized it for some
internal trainings.
(00:21):
Yeah.
Yeah.
And um that so that spawned likea whole like uh what's going on?
And it's like, hey, let's thisis a third party, you guys are
okay, this is what this means,kind of thing.
But it's kind of interestingthat they had no idea, and this
had been already, you know, inthe news for a while.
SPEAKER_00 (00:42):
You are now listening to the Secure AF
Podcast.
SPEAKER_02 (00:51):
Welcome to another episode of the Secure AF
Podcast.
Uh, I will be your host today,Andrew, uh security engineering
lead here at AliasCybersecurity.
And joining me today is mycolleague and mentor, uh,
Jonathan Kemet.
How are you doing, sir?
Mentor.
I like this.
Yeah, I was I would call youmentor for sure.
He's our CISO here at AliasCybersecurity.
SPEAKER_01 (01:11):
Uh I only teach good things sometimes.
SPEAKER_02 (01:14):
Usually, yeah.
Um, and today we're going to betalking about a subject that is
all over uh cybersecurityheadlines today for a good
reason.
And that is the Canvas uhplatform.
Would you would you call it adata breach?
Would we call it a breach atthis point?
SPEAKER_01 (01:32):
I I would say yeah, it's a breach.
Incidentally, in most cases, uhthe breach would require
notification, and this is goingto require notification.
Yep.
SPEAKER_02 (01:42):
Yeah, for sure.
That's fair.
Um, and this uh breach incident,breach, um, it affected a lot of
organizations.
Yes, a lot of organizations.
So uh it's just to start out,uh, for listeners who may not be
familiar with what exactly theCanvas platform is, um, who tip
(02:04):
who typically uses so Canvas isa learning management system.
SPEAKER_01 (02:09):
Um, so when you imagine someone going to school
and their instructors areputting up content for them up
online that they can go in andread or watch or submit uh
papers or submit stuff, uhthat's through the learning
management system.
And and generally speaking,there's three or four large
learning management systemsthroughout all of education.
(02:32):
Uh, Canvas is one of them.
It's a company calledInfrastructure and no
Instructure.
Um, I always get that messed up.
I always say Canvas, that's howeveryone calls it.
Uh, but the company is calledInstructure.
And uh it was it's it's verywell used.
I mean, and it's not just uhhigher ed or K through 12, a lot
(02:55):
of organizations use the sametool within their own training
programs, right?
Uh, and using it as a learningmanagement system.
That's what it's designed for.
Um, when you go on theirwebsite, you're gonna see some
really big names at the bottom.
There's a lot of people who useit.
SPEAKER_02 (03:09):
Yeah, yeah.
And I mean, my mother-in-law,who's a librarian at elementary
school here, you know, we wereover at their place yesterday
and she said, Okay, whathappened with Canvas?
Uh so I mean, a lot oforganizations, uh higher
education, elementary, like theywere down.
SPEAKER_01 (03:28):
Yeah.
It was hard down.
It was hard down for a littlewhile.
It's we've been having issueswith it for about a week all
last week.
Uh, but they took it completelyoffline Thursday afternoon and
it came back up in the middle ofthe night.
SPEAKER_02 (03:42):
Right.
So, how critical is Canvas forday-to-day operations for those
organizations like universities,enterprises, or just training
programs?
SPEAKER_01 (03:52):
Um, well, for universities in some K through
12s, they were hard down becauseum one of the things that
happened during COVID is schoolsystems started pushing all
their classes online.
And at that time, they chose thedifferent learning management
systems that they did.
Some did Blackboard, some didD2L, some did Canvas, you know,
(04:15):
whatever it was.
And so they they, in fact, Iknow some schools had a specific
mandate that every one of theirclasses would have
representation in the learningmanagement system.
Well, what that turned into isthat is how the class is run.
The that is how the students getin there, that's how the faculty
get in there and get theirstudent roster, that's how
(04:36):
students uh get their content,that's how they submit their
content.
It is really that's how wheretheir grades come from, their
grades get downloaded from theirthe the LMS into their student
information system.
I mean, it's critical.
Right.
Um, the the afternoon it wasdown, the hard down.
Uh there was universities thatliterally just shut down
(04:57):
classes.
They didn't, they couldn'tsubmit anything, they couldn't
do anything.
Now, one of the problems forthis particular incident and
this particular timing is it wasfinals.
You know, it was the last coupleof weeks of school.
This hit them hard.
They were trying to get thoselast grades in, and it was it
was very impactful.
SPEAKER_02 (05:16):
Right.
And that kind of launches meinto my last question here uh on
the setting the stage portion.
Um, from a CISO lens, how wouldyou classify Canvas, like
mission critical, sensitive, orboth?
SPEAKER_01 (05:30):
Oh, I I absolutely call it critical infrastructure.
Critical infrastructure for theorganizations that are using it
to that degree, it would beconsidered uh critical
infrastructure, right?
Critical systems.
Uh, because the primary role ofeducation, you know, K through
12 or higher ed is to provideacademic services to teach
(05:52):
students.
And if their their only way toteach is through that learning
management system and it's down,that's a problem.
Now, when I was going to school,we did have a version of
learning management.
Now, this was a very long timeago.
I got lots of gray.
We had a version of learningmanagement, but we only had a
handful of classes in it.
Right.
When I was at the university,when I was a CISO there, we
(06:16):
probably had a half to threequarters of our classes in it.
But again, after COVID, becauseof the what happened at COVID,
everyone had to have an onlinepresence.
Well, learning managementsystems was that online
presence, so they just movedeverything into it.
So now, I mean, there areinstitutions that don't have any
class that is not in the LMS tosome degree, and most of their
(06:40):
classes are entirely in the LMS.
That means their content, theydon't they don't even have
PowerPoint presentationsanymore, like outside of the the
the LMS.
So it's not like you're going toa class and they bring a
computer up and a PowerPoint.
It's they log in and all thatcontent is there.
So it is absolutely critical forthose institutions.
SPEAKER_02 (06:58):
Yeah, and that's another perfect transition into
talking about the dependency andlike risk surface here.
Oh, yeah.
So um you mentioned, you know,during COVID, there was a huge
shift um just trying to survivein most cases.
Yep.
Um, and that gets into thequestion of how heavily are
these organizations relying onon SaaS platforms like Canvas
(07:22):
compared to on-prem systemstoday.
SPEAKER_01 (07:25):
It's all SaaS.
I mean, most of theorganizations are moving to they
either have moved uh because ofCOVID andor other things, or
they are moving.
Um, and you know, when we thinkabout higher eds or even K
through 12s, and what we'retalking about is either
Microsoft or Google.
Yeah um they're running an LMSof some kind, their um their ERP
(07:49):
and student information systemis probably SaaS.
Some institutions still haveon-prem things, but a lot of the
organ or a lot of the providersare saying, we're not gonna
support that in the next twoyears.
You have to move off to our SaaSplatform, which makes sense from
that vendor perspective becausethey don't want to keep
maintaining old products, oldon-prem things.
(08:12):
So they're just saying, okay,we're not gonna do any more
updates to it.
You know, you don't have tomove, but you're not gonna be
using the service anymore.
So I would say mostorganizations probably have 70
to 90 percent in some version ofa SaaS.
Right.
Um for those that aren't,they're getting there.
(08:35):
I mean, they're they're they'reworking to that.
I would say in five years, uhprobably 90 to 95% is all gonna
be SaaS products.
SPEAKER_02 (08:43):
So what risks come with centralizing so much
operational and user data into asingle LMS platform?
SPEAKER_01 (08:53):
Well, the risks there's a couple different
risks.
Number one, you have no controlover it.
Right.
SPEAKER_02 (08:58):
You're outsourcing that security.
SPEAKER_01 (08:59):
Yeah, you you're everything is on that vendor.
So you've got to have the rightvendor management program, the
right contracts, the rightcheck-ins, all those different
things.
But then you're just prayingthat the vendor does good stuff.
Right.
I mean, we saw in thissituation, you know, we've had
9,000 institutions go down, orsomewhere around that 9,000.
(09:22):
There is nothing they could havedone.
There is absolutely, and I'vetried to get this across to the
institutions, there was nothingthey could have done any
different, unless we talk aboutbusiness continuity.
That's a little different.
But in terms of the the attack,the threat, the threat profile,
the vulnerabilities, that wasall on the vendor.
(09:44):
The vendor is the one thatmanages it.
So when you're looking at risk,that is high impact, hopefully
low probability.
So when you're doing your riskmatrix, the impact versus
probability, um, it's highimpact for sure, hopefully low
probability, uh, but it's notzero.
And in this particular case, itwas very clearly the vendor had
(10:07):
a problem, it had a huge impacton the institution.
Now, for higher eds, there'sonly a few things that really
have a negative impact shortterm.
You can affect higher ed longterm, but let's say you had the
internet go down, you know, atthe institution, probably a
couple of days before itactually caused harm of some uh
(10:30):
monetary harm or something else.
Um, but that's not the way withLMS when all of your teaching
capability is in that tool andit goes down, even just for us
in our tool, that is your entirebusiness.
So that one is huge, hugeimpact.
(10:51):
So you do a couple things.
I mean, you have to evaluatethat risk just like that.
What is the impact?
Huge.
What is the probability?
Probably pretty small, but notzero.
So you have to put in yourbusiness continuity in to make
sure that you take that intoaccount.
Hopefully, it never happens, butwe saw here it did.
So, and and I think there's alot of work, and this is as far
(11:12):
as I can remember, this is thefirst time that we've had a
major LMS outage like this.
I think so.
Um, and it no one was expectingit.
Yeah.
SPEAKER_02 (11:23):
So, in your experience, are organizations
generally overconfident in theseSaaS providers' security
controls?
SPEAKER_01 (11:33):
That's a good question.
I don't think they'reovercompetent.
I think they just don't know.
They don't know.
I mean, I you know, I I workwith organizations all the time
in their vendor managementprotocols, and they just they
don't ask questions, they don'thave an MSA in there as it
relates to uptime.
You know, you need to have anSLA, in some cases, OLAs, you've
got to have um, you know, whatis it that they're doing in
(11:55):
terms of their implementation ofsecurity controls?
What are they expecting risks tobe?
Now, you can't prepare foreverything.
I mean, this it's not evenreasonable to think that.
But some of the more commonthings you want to have an
alternative for.
So uh I can almost guarantee youthat from this point forward,
(12:15):
whenever someone is asking, youknow, what are your security
controls?
Another question they're gonnaask any LMS, or at least I hope
they do, is what happens if yoursystem goes down?
How do I get my data so I canfinish my semester?
How do I finish finals?
How do I do this?
What options are we going tohave?
And I'll tell you, no one has agood answer to that yet.
SPEAKER_02 (12:38):
All right, so let's talk about the attack itself.
All right.
Okay.
So at high level, uh, what do weknow so far about this canvas
breach?
SPEAKER_01 (12:50):
So, what we know is based off what we're being told.
Uh, so make sure that everyoneunderstands that.
So this is coming from Canvasand from what we have found out
on the internet for as much asthat's worth.
So we believe that the attackeris called Shiny Hunter, um, or
(13:11):
that is the attacker that didthe attack.
Uh, from the company, it wasthere's a tool called Free for
Teacher that Canvas provides.
And this is not their normallicensing structure for higher
rate institutions or K through12, it's something else, but
it's for a teacher that may nothave a big license, they could
(13:35):
get into this tool and use itand create for their classes.
Um, what we're being told fromthe company is there was a
problem within that tool thatallowed the attackers to get in
and then do effectively aprivesque or some version of
vulnerability to get into theback end systems of that tool.
(13:56):
So we don't know a lot ofdetails that hasn't been
released.
Uh, as of today, um, this wouldbe 5-11 of 2026.
I know that the uh person whodoes our podcast may not like me
saying that out loud, but thereality is it's important here.
As of today, we know theattackers got into that system
(14:17):
and the company is running anincident response investigation
to figure out what all hashappened.
And they said that they're goingto release at least parts of
that to the customer base.
So hopefully in the next coupleof days, we're going to have
more information about what theydo know and how that has
affected the rest of thesystems.
SPEAKER_02 (14:38):
Yeah, and I think right now they're saying that
the exposed data based onconfirmed statements or names,
email addresses, uh, studentIDs, and uh chats in chat.
So that would be internalmessages between Canvas users.
Yes, that's what they'rereporting.
SPEAKER_01 (14:56):
That's that is what they're reporting.
Um now I would say this that theDepartment of Education still
counts that information to beFERPA information.
So um there will always be aspin on this on any sort of
announcement out to the worldabout what is what is out there
(15:17):
and what you're going to need todo.
And you know, people will try todownplay as much as they can.
I mean, they're saying thingssuch as um there was no social
security numbers, there was nocredit card numbers, there was
no okay, great, but theinformation that was released,
um, the name, the the IDnumbers, the chats that can
still be considered FERPA data.
(15:37):
So there is still requirementsthat organizations are gonna
need to do uh breachnotifications, DOE
notifications, all kinds ofdifferent things.
So, again, this is what we'rebeing told.
Yeah, um, more information willcome out in the future after the
investigation is complete orafter they you know do as much
as they can.
(15:58):
Um, and I believe that the enddate for the threat actor is
tomorrow to release theinformation.
Now, one of the things that youknow we do this a lot in terms
of working with the attackersand working with the
organizations, you know, thatthe that middle piece there,
sometimes we don't knoweverything until they've
(16:21):
released the data.
And then sometimes we releasethey release the data and like,
oh, that was a lot more thanwhat we were expecting.
So or we were even seeingexactly.
So we don't know right now, wedon't know everything.
So as soon as the attackers, ifthey're going to release the
data, then we'll have at leastthat information to some extent
(16:43):
after they finish theinvestigation at the
organization level at the canvaslevel, we might have some more
information there.
We just don't know right now forsure.
SPEAKER_02 (16:52):
Gotcha.
So do we know if the motivationbehind this attack?
Like, was it financial?
Was it um espionage hacktivismor opportunistic?
SPEAKER_01 (17:04):
Um, I I think it was financial.
Um, I mean, I think that they'vecontacted them saying, hey, uh,
give us money, we're gonna getrid of your data.
Um I there has been reports thatindividual users have been
reached out to.
We don't know if these arescams, we don't know if it's
really the threat actor, butagain, it's a uh give us money
(17:25):
and we'll delete your data typething.
So I think at the moment, webelieve it is a financial
attack.
SPEAKER_02 (17:32):
Do we know how this breach was initially detected?
Like did the provider come out,or was this like the customers
were going, hey, we're canaccess our stuff.
We're seeing this pop-up.
What's happened?
SPEAKER_01 (17:44):
Um, so I've heard a couple of different things.
Some people there the the thethreat actor did post
information on websites, yeah,you know, that they had gotten
into it.
Um there was supposedlyinformation provided to them,
like a ransomware note or uhsome sort of communication
saying, Hey, we got in give usyour money.
(18:05):
Um, to my knowledge, at leastnone of my clients ever saw
anything from a usabilityperspective.
Nothing changed in theirversions or in their client or
in their uh in their webinterfaces.
They didn't see any of that.
They're going off what they werebeing told by the leadership by
from Canvas and by what they'reseeing in the news.
SPEAKER_02 (18:27):
Now I had a customer that reached out, one of our
customers reached out to me witha screenshot saying, uh, hey, we
had a user navigate to thistraining website that we use,
and there's this shiny hunters,and we'll take a screenshot and
send it to us.
And they were super concerned,and this wasn't even a higher
ed.
Yeah, this was an organizationthat just utilized it for some
(18:47):
internal trainings.
Yeah, yeah.
And um, that so that spawnedlike a whole like what's going
on?
And it's like, hey, let's thisis a third party, you guys are
okay, this is what this means,kind of thing.
But it's kind of interestingthat they had no idea, and this
had been already you know in thenews for a while.
Yeah.
SPEAKER_01 (19:07):
Um well, just like any other incident, people are
gonna try to downplay it becausethey don't want notoriety.
Yeah, um, and in this situation,the the tool itself is used so
widely, it's also wrapped up inother names.
So, I mean, they'll take thefunctionality and they'll put a
different name on it for thatorganization or brand it however
(19:29):
they need to.
So people may not have knownthat it was Canvas, right?
Because that was not how theycall it.
It's just an LMS to them.
Um, but most of my people that Italked to, their first notice
was a notice from Canvas thatsomething had happened, which
you know that came out, Ibelieve, on May 1st uh was the
(19:51):
first one that we saw.
And my talking about my client.
And uh there was three or fournotices after that.
Every every other day or so, wehad gotten notices with a little
bit more information about it.
But of course, we saw stuff onthe news, there's stuff from
other organizations, uh, stuffon X.
I mean, there's all kinds ofdifferent areas where people
(20:11):
were posting things, which canmake it difficult because you
don't ever know what is truthfuland what's not.
Even some of the information wewere getting from the
organization, it's not that theywere lying, they just didn't
know.
And so you you don't want toinfer anything or assume
anything.
You just you're trying to workwith the best information you
(20:32):
have at the time.
For sure.
SPEAKER_02 (20:34):
And so what does this incident tell us about uh
detecting blind spots in SaaSenvironments as an organization?
SPEAKER_01 (20:43):
Well, it's really hard.
It's really hard within the SaaSbecause you have to rely on the
SaaS to do that.
But the blind spots for me as aCISO is going to be business
continuity.
If this system is not available,um, how do we function?
SPEAKER_02 (20:58):
And I think going beyond just this event, that is
a common problem or common justtheme that we see doing as in a
response, where they have thistool that they utilize that
they've invested 100% of intozero backup strategy.
Like, no, like, yeah, hey, ifour one tool goes down or breaks
(21:23):
or disappears tomorrow, but butthat's not how it's sold.
SPEAKER_01 (21:28):
Right.
I mean, when you think about it,software as a service was sold
as we've got all these datacenters and we have all this
redundancy, we're always up.
Right.
I mean, think about AWS or Azureor anything, that that's how
it's sold is you know, you havefull redundancy, so we're always
up.
(21:49):
So we the the customers havebeen sold this, it's just and
and the salesperson may not haveever said it, right?
But the fact that you'rethinking as a That's how they're
thinking about it.
For sure.
So, and I'll and I'll tell you,Canvas was not down because of a
(22:10):
resource limitation.
Right.
It was not down because theirservers went down, it was down
because they turned it off.
They they went into amaintenance point, which don't
get me wrong, I think it was theright thing to do for them.
They were turning it off to getthe attacker out.
100% agree with that.
If I was there, that's probablywhat I would have done as well.
You know, and I don't want tosecond guess the CISO there.
(22:31):
You know, I'm sure that theywere all running, you know,
crazy just to get thingsfunctional.
But all we know is theyintentionally turned it off.
Now, going back to that riskthing, how do you know your your
vendor's not going to do thatfor you and you have no control
over it?
Right.
Now, even if they had an SLAsaying they won't do that, or
(22:56):
they'll have 99.999%, you know,five nines uptime, I mean, some
bad things happen, and the bestthing to do is turn it off.
Um you can't get around that.
And since you don't have anycontrol over it, all you what
you can do as the institution isokay, how do I continue
(23:17):
providing the functions?
Now, let's be real about this.
This was higher ed and K through12s, and this was learning.
This is academic stuff, and itwas very impactful to them.
For sure.
But it wasn't a hospital thatwas trying to provide patient
treatment, it wasn'tinfrastructure, you know, oil
(23:37):
and gas and electricity andwater and things like that.
That when it's down, you'reyou're talking minutes before
impact is felt.
So huge impact for sure, but itwasn't as bad as maybe another
industry they got here.
Or critical industry.
Right.
So, I mean, we got to balanceit.
(23:58):
Um, for my clients, you know,I'm gonna recommend, I am
recommending that we doimplement some better business
continuity.
How do we teach these things?
How do we still provide serviceseven if the SaaS goes down?
I'm not saying don't use it.
I'm just saying, well, let's,what are our alternatives?
Right.
You know, can we do somethingdifferent?
(24:21):
Maybe we have a backup of somecontent that, well, we'll just
get them online and we'll do itanyway.
You know, I I'll say this youknow, one of my clients had it
was a major impact.
Of course, it's the time ofyear.
And uh we were talking, it waslate one night that we were
still working on this, seven oreight, nine, ten o'clock,
whenever it was.
(24:41):
And one of the things I got themto understand is we don't make
any decisions right now.
You know, they had just turnedit off.
You know, Canvas had just turnedthe systems off.
Uh, let's say four o'clock,whenever it was.
I don't remember now.
And so we were planning, youknow, okay, what if it doesn't
come back on?
You know, what's worst casescenario?
(25:02):
It's not coming back on.
We have faculty members leaving,we have students trying to
graduate.
You know, what what is it we'regoing to have to do?
It's like, okay, I 100% agreewith thinking about this.
Don't make any changes.
Right.
Don't make any changes becausewe don't know yet.
And let's see what they do.
(25:23):
They're affecting 9,000institutions.
If they were going to be downfor a full day, they would
absolutely lose market share.
SPEAKER_02 (25:32):
Right.
SPEAKER_01 (25:33):
So everything came back on about 11 or 12, is what
we think.
Uh, we got notices at aboutthree in the morning saying
everything was back up.
And they were able to go throughthe next day fully functional.
Now we didn't know much, but weknew that the tool was
functional and were able tofinish out the semester, finish
out that week.
(25:54):
Um I do appreciate them notbeing down long, even though I
understand why they were down,but it allowed the institution
to be able to finish out thelast little pieces that they
needed.
SPEAKER_02 (26:08):
Yeah.
So we've talked about theinitial impact to organizations,
but what kind of downstreamrisks are we going to see from
this?
Um, and I'm thinking about, youknow, based on what we know,
what data may have left, uh, orpossibly more, kind of depending
on what they gain access to.
Uh, but down downstream risk,kind of thinking about like
(26:31):
credential reuse, um, any kindof future phishing campaigns,
yep.
Uh, or uh identity theft.
SPEAKER_01 (26:37):
Like, I I think all of those are absolutely going to
happen.
You know, I think students, thestudents' names and their emails
are going to are out there now.
Um, I think attackers or maybecopycat attackers are going to
say, hey, give us X amount ofmoney and we'll delete your
data.
You know, they might say We'veseen that in the past.
Absolutely, we have.
Um, so I think that that isthat's going to happen.
(26:58):
Um, I think one of the thingsthat no one is really talking
about right now, but when ShinyHunter did this, this is huge
impact for higher ed.
I think other LMSs are going tobe targeted much more heavily
now because now they saw whatthe impact was.
So I think they're going to goafter them.
(27:19):
So if I was an LMS group, Iwould be absolutely learning
from this.
And I would be making sure thatthere you've got segmentation,
you've got MFA, you've gotanything and everything you can
think of to deploy in yourenvironment to prevent something
like this, and have strategiesfor if it were to happen.
(27:39):
So I think that will absolutelyhappen.
Um, yeah, maybe have some pentesters come in.
Yes, pen testers and riskassessments now it's and
contract management.
I some of these contracts arepretty bad and they don't have
SLAs in them, they don't haveany of these things.
So I think that those are goingto be, you know, on the uptick.
The other thing that we saw fromanother data breach from about a
(28:02):
year and a half ago wasinformation that was gathered in
that data breach was used tocreate fake accounts in the
higher ed, which was then usedto get federal financial aid.
So it was uh financial aidfraud, but it was real students.
It was the real studentinformation.
So I think that as that gets outthere, you might see an uptick
(28:23):
in fraudulent applications intoinstitutions trying to get in to
get a user account to be usedfor other things, or to even
apply for federal financial aid.
SPEAKER_02 (28:31):
Absolutely.
Yeah, I didn't think about thatone.
So that's a really good pointthere.
So let's get into your favoritetopic of this.
What kind of regulatory orcompliance issues are these
affected organizations going tobe looking at?
Well, they have to report.
SPEAKER_01 (28:50):
I mean, no matter what this is, even though this
is a third-party vendor, you area controller.
So a higher institution, eventhough this is your vendor that
got compromised, you're thecontroller of the data.
You have to do the reporting.
You have to report it to theregulatory side, you're gonna
have to report it to the usersas a hitch breach notification
laws, all the state laws andeven country laws, meaning if
(29:12):
you have an internationalstudent base, you still have to
report this.
Now, the reporting is prettysimple.
Hey, our vendor got hit.
Now, the DOE absolutely knowsabout this.
They are still going to want youto report because it's your
responsibility to report thisappropriately.
You will have to do the briefnotification, so you're gonna
have to let your students know.
(29:33):
And you have to say, it wasn'tus, it was our vendor, but we're
letting you know what thishappened because there still has
to be a risk assessment done atthat student level.
The student has to look at itand go, okay, well, maybe I am
going to see a lot more phishingattempts.
Maybe I am going to see thisstuff, maybe you know, password
reuse, whatever.
So they're gonna have to do arisk assessment of themselves if
(29:54):
they know it happened.
Now, I think everyone prettymuch knows it happened, but you
have to do those officialnotifications.
You have to document it, youhave to get it out to them, you
have to you use your bestprocess to get that out there.
Um, you also will have to doremediation processes.
How do you prevent this in thefuture?
Now, this is vendor management.
(30:15):
It's the vendor that had theproblem, but you have to do
vendor management processes tohelp mitigate this problem in
the future.
So I would absolutely be usingthis as a tabletop example, as
some process that you have aplaybook for of what happens
when it happens again.
What did we learn from it?
How do we resolve this?
You know, what is the timing?
(30:36):
Maybe you have to if you have afour-hour response time from
your vendor to let you know thatthere's an issue, well, then you
can have a six hour before yousend out a note to your campus.
So, what does your playbook sayabout this?
This happened.
Now, I put stuff like this on mytabletops all the time, and
everyone's like, Oh, that willnever happen.
(30:57):
It did happen.
Yeah.
So let's build a playbookbecause it will probably happen
again.
SPEAKER_02 (31:02):
That's funny to me on the the several tabletops
that I've been with you, which Ilove doing on.
Uh, I love your hey, you're nothere, or you got hit by a train
coming in and you're not here.
But how many times have we beenon an incident and the guy we
need to talk to is on vacation?
Arupa.
SPEAKER_01 (31:19):
Yeah.
Well, we just had one where thethe uh a stakeholder that we
needed to talk to was on acruise ship.
Yeah.
It's like, okay, well, that'sout.
Yeah.
SPEAKER_02 (31:28):
Yeah.
So you just, you know, you gottahave backup plans, you gotta
have your chain of command umfor sure.
So we've kind of talked aboutlike where the responsibility
falls between uh the platformprovider and the customer.
So, what should organizationsrealistically expect from their
SaaS providers uh just in termsof security and transparency?
SPEAKER_01 (31:51):
Well, okay.
I'm gonna say it a very specificway, they should expect what the
contract says.
Right.
Now, you could have elevatedexpectations depending on if
it's a critical function or not,but whatever that expectation
is, it needs to be in thecontract.
So if it's a certain amount oftime before they report it to
(32:11):
you or they report to youcertain information, whatever it
is, you want to make sure thatis in the contract.
Now, for higher ed, uh, and evensome for uh K through 12, uh,
you have a 24-hour requirementbefore you have to report things
to the Department of Ed and theState Department of Ed and all
kinds of different things.
So 24 hours.
So you really need to knowsomething of a suspected
(32:35):
incident within that 24 hours.
That needs to be in thecontract.
One of the other things that wefound out is we one of my
clients, we didn't know who thenotifications were going to.
Right.
So the organization, Canvas inthis case or instructor, was
sending out notifications.
(32:55):
We didn't know who was supposedto receive those.
Now, sometimes it's the personwho signed it, sometimes it's a
specific person, but is thatspecific person still there?
So those sorts of things comeinto play.
So you need to know what yourcontract says, and then when a
situation happens like this,read your contract because that
(33:16):
contract is going to tell youwhat to expect, when to expect
it, and if you don't like it,get a new contract.
Right.
When it renews, you make surethat stuff is in there.
Contract management is huge, andI I harp on this a lot because
we have been in incidents whereI asked to see the contract, and
they're like, Oh, we don't haveit.
Then you have no expectation ofanything at that point.
(33:39):
You need to put thoseexpectations in before you need
them.
So that's what I would say isyou're going to get hopefully
what the contract says.
Sometimes not.
Sometimes they will not give youthat information.
Because here's kind of a redflag there, though.
Well, but I mean, we have seenit ourselves where people are so
(34:01):
tied up, and you know, you'reyou you've got a system scene.
It started on May 1st in termsof the first notification.
I'm sure their teams wereworking 24-7 for six or seven
days straight.
Right.
They're not thinking about thecommunication side, they're just
trying to get it figured out.
So sometimes they don't.
I I I've got one story, Iusually do a lot of stories, but
(34:24):
this is the one story.
We had a data breach of a SaaS,and we were notified like three
months later.
Woof.
Um, it went to someone thatdidn't really know what to do
with it, so they ignored theemail in terms of coming into
the organization.
An IT person just literallyhappened to see it on the screen
(34:48):
when he remoted into hercomputer and says, That's odd.
Why what is that email?
So that finally got to me.
I'm reading it and going, Ohgod, what what no one told us
about this?
Well, skip ahead.
Um, three weeks.
We were very early in terms ofcommunicating with the that
(35:10):
team, and I was able to actuallyget a meeting with their CISO.
Now, this would be like gettinga meeting with the CISO at
Canvas, right?
Really difficult, but I I wasjust fortunate.
I had lots of lawyers on ourside, lots of lawyers on our
side their side, but I got totalk to him.
So we got on a phone call.
I had about 10 minutes, I askedhim a bunch of questions.
(35:33):
He actually did a good job.
Yeah, everything he did wasright.
I I was like, okay, I appreciatethat.
He did what was right, Iunderstand why he did, why he
did it.
It was fine.
Well, he got off, and then theCEO for that division got on,
and the question, the finalquestion from the our lawyers
(35:57):
was okay, the contract, the thecontract says you will tell us
within three days.
It took you like 60 or 90 daysor whatever it was to notify us.
Why that is a completelylegitimate question.
Yeah, contract says this, youdidn't do this.
And his response, now I'm I'moff at this point.
(36:21):
I've my my camera's off, my micis off, I'm done.
And he goes, Well, I'm reallyglad you asked that question
because I've been doing myresearch and it doesn't look
like anybody ever meets thoserequirements in that contract.
And then he turns to me and hegoes, Well, Jonathan, what do
you think about that?
(36:44):
I I turn my camera on and Iturned on my uh my mic and go,
Well, I'm really glad you askedthat because I teach a contracts
course for cybersecurityprofessionals at various
conferences.
And so my question back to youis if we don't have to follow
this piece of the contract, dowe have to follow any of the
(37:04):
contract?
Right.
They didn't like that question.
Of course not.
And their lawyers cut the call.
Just nope, fuck.
So the contract is reallyimportant, and I harp on this.
I mean, people make fun of mebecause I like reading
contracts.
Well, the reason I like readingcontracts is I don't want to
(37:24):
ever get caught off guard.
I want to know exactly what isin that contract, what we have
to do, what we're supposed todo, what we don't have to do.
Your contract, your masterservice agreement, whatever you
call it, is critical.
You spend the time up front sothat when you have problems
later, you have a contractthat's going to back you.
Yeah.
SPEAKER_02 (37:44):
It's part of the having a plan.
Exactly.
Exactly.
So we've discussed a lot aboutlessons learned already just
through this conversation.
But what role would like loggingand monitoring play here,
especially when visibility islimited?
SPEAKER_01 (38:02):
Um, so for my clients, what I told them in the
very beginning is review yourlogging to see if because we
didn't know anything, see ifanything was coming back.
Anything was affecting oursystems that looks odd.
Anything in terms of the datacoming from Canvas back to us
looks odd.
(38:22):
So we were trying, I mean, yourlogging is you is you when you
don't know what's going on, yourlogging is key.
I mean, that's how you deal withthis.
So um, in the very first day,first hours, we were absolutely
looking at that logging.
Yeah.
Um, we couldn't tell anythingbecause it wasn't that sort of
an attack, but we didn't knowthat.
All we know is that there was aproblem.
(38:44):
Our number one priority is toprotect our systems.
And that's where we went in.
We started looking at the logsof our systems to see if any
problems, any user accounts, anydata, anything looked odd so
that we could deal with it.
SPEAKER_02 (38:58):
Right.
So let's let's talk about somepractical recommendations for
security.
Okay.
Some of them we've alreadytouched on, um, like talking
about like what theseorganizations could do or what
steps they would take if uh theyuse Canvas or similar platforms.
But how could an organizationstrengthen their incident
(39:20):
response plans for third-partybreaches like this?
SPEAKER_01 (39:24):
Well, first they need to realize that it is the
risk.
I mean, what risks there are.
So you need to plan that in, youhave a risk assessment around
that.
You need to, of course, havethat the contract side of what's
going to happen during, youknow, the during the situation
where how they're going tocommunicate.
Um, but it really does come downto business continuity.
I mean, you need to understandthat if the risk, here's the
(39:48):
risk.
A lot of people will look at therisk of the data got
compromised, and that's bad, andwe have to deal with that.
But the other risk is youcouldn't teach classes, right?
You couldn't graduate, youcouldn't do this.
One of the things I asked one ofour clients was, What did when
you can't do a final, how do youdeal with that?
(40:10):
Do you postpone it till nextweek?
Do you just say, Well, whatevergrade you get is what you got,
right?
Um, what is it?
And they were like, We've nevermissed a final.
I'm like, I know we missedfinals during COVID.
SPEAKER_02 (40:22):
So but my question is, these or I mean, Canvas
hasn't been around forever,right?
These organizations had beendoing I mean LMS's have been
around for probably 20 years.
Yeah, I know, but before I meanschools have been around longer
than 20 years.
Well, well, just what were theybefore?
You know, this is this gets intolike my like I don't know why I
(40:44):
think just having like the thesimplest backup plan is the most
effective.
Like what how did we do it 20years ago before the LMS?
SPEAKER_01 (40:54):
Well, but but here's the difference.
I mean, I understand the backupplan, but schools can't be
redundant, they don't have theresources to be redundant.
So if you have all of yourcontent in your LMS, you're not
gonna maintain your contentsomewhere else.
That's redundant, right?
That's that's more effort, morebackup, more whatever.
The whole intent is to get it towhere the the tool itself is the
(41:17):
redundancy.
In this particular case, theytook that down.
So I get it.
Um, and one of my questions waswell, why can't you just do a
Zoom call and teach anyway?
Yeah, I mean, you have thecontent, you should know what
the syllabus should say, whatthe content is.
Well, yeah, the syllabus was inthe LMS.
They didn't have the syllabus,they didn't know what content.
Now, this was finals, so thisthey weren't actually teaching
(41:38):
like that.
They were trying to get theirfiles in and get their grades
out of there.
Uh, but I will say I had oneclient who uh they still
maintain gradebooks.
Yeah, yeah.
If for those of us who are old,the old grade books that were
green and they had the lines inthere, they still maintain
those.
So they had all the grades.
Now we had a way to get thosegrades into this the student
(42:00):
information system, um, but veryfew people have gradebooks
anymore.
Yeah.
So it that's the thing is you'vegot to have the backups,
absolutely, but they don't havethe resources to be redundant,
so a lot of them don't do that.
Okay.
SPEAKER_02 (42:14):
Let's get into executive and board level
perspectives.
Okay.
Because I know you're gonna havesome great answers here.
Because these are theconversations I don't want to be
in.
Yeah.
So, how should CISOs communicateincidents like this to
executives and the boards?
SPEAKER_01 (42:31):
If this is the first time your board is hearing about
your issue, yeah, let's say thisis the very first time.
Yeah, this is this is a problem.
Um your board, when you'retalking to your board, they
should not, you should not bethey should not be surprised
that a vendor going down hascaused such an impact to the
(42:54):
organization.
That should be in your riskregister, right?
People should know, oh, theseare critical systems.
If they go down, criticalfunctions fail.
So uh if you've never done thatbefore, uh, you're gonna take a
hit on the chin.
Not because they went down, butbecause people were not aware of
(43:14):
it.
So communication here is key.
You need to get in early, youneed to explain to the board
here are all the systems that,if they go down, have a negative
impact on us being able tofunction.
They need to be aware of that.
Now they're gonna ask you, well,what are you doing to mitigate
that risk?
And some things you can do andsome things you can't do, but
(43:36):
that's a risk appetite issue ofwhat will the board allow you to
do?
Now, the board says, Nope, youhave to have full redundancy of
your LMS.
Okay, so we're gonna have twoone is Blackboard, one is
Canvas, right?
And you're gonna load your yourclasses in both, and people can
go to both, and that's really,really difficult in higher ed.
(43:58):
Absolutely.
So
SPEAKER_02 (44:00):
Then you have people who are like, I want to go do
this and it's not here.
Exactly.
It's like you can't hold meaccountable for that.
SPEAKER_01 (44:06):
Right.
Your system is messed up.
But I think it is fair to let Imean, you have to let the board
know that this is a risk.
Right.
Now, if they know it's a riskand it happens, what you're
doing is you're going to theboard quickly and going, Okay,
here's where we're at.
This is what we're communicatedwith.
This is what our expectationsare based off the contract.
Here's where we're at in termsof notifications and
(44:27):
communications with legal, withregulatory, with whatever.
You have your playbook ready togo.
You should have a playbook forevery single critical function
you do.
That if it's not there, what areyou doing to keep the business
running?
Right.
Now, if you have that playbook,you're walking in with that
playbook.
You walk in there, you go,here's what we know.
Here's everything that they'vesent us.
(44:49):
This is what we've seen in ours.
Here's what we're doing nextwith the playbook.
I think that is the mostcritical piece that any system,
any security professional or ACso when they walk into that
board, they need to have thatstuff ready to go.
If you just walk in gettingready to take a hit, it's like,
oh, well, I don't know anything,but I'll go in there.
Well, yeah, you're gonna take ahit.
(45:10):
And you're probably not gonna beasked back because if you're not
providing value to the board,right, they're not gonna have
anything.
Right.
So that's what I would say isyou should prepare the board
beforehand, that these arecritical functions.
I'm talking months, years ahead.
They should be completely awareof what your critical functions
are.
And if Canvas goes offline, thisis the problem.
(45:33):
That way they know about it.
And they're like, okay, got it.
We understand that's critical.
We understand we can't doanything about that.
Well, we will continue on.
SPEAKER_02 (45:42):
Yeah, it just gets into understanding the risks.
SPEAKER_01 (45:45):
Exactly.
SPEAKER_02 (45:46):
Exactly.
Have there been any risk framingthat resonated most when
discussing these third-party uhbreaches with your clients?
SPEAKER_01 (45:58):
Of the two that I I always go after is the
regulatory side and the impactto the users.
Right.
You know, what what is going tobe the impact to the students in
this?
Not from the fact they couldn'tget their grades in.
What is going to be the impactof that data being lost?
You know, I I'm I will say it,you know, every single
presentation, our job is toprotect people.
(46:19):
So I always make sure that whenwe have a data breach, I make
the leadership completely awareof what risks we have to the
organization for sure.
But also what are the risks tothe customers or the students or
the the patients, whatever?
Because I I think that is reallyimportant for us to understand.
For sure.
SPEAKER_02 (46:38):
All right.
So let's get into some closingthoughts now.
Okay.
Uh couple questions.
One, what's one thing everysecurity leader or Cisho CISO
that should what should theytake away from this incident?
SPEAKER_01 (46:54):
This is a vendor.
I mean, this is what happens.
You put that much trust in avendor, a vendor can absolutely
take down your business.
SPEAKER_02 (47:02):
What should an organization be doing today to
prepare for the next SaaSbreach?
SPEAKER_01 (47:07):
You need to understand all of your SaaS,
what are critical functions, andthen what is your business
continuity around thosebusiness, those business
functions, those criticalfunctions that will make sure
you can continue to work even ifthis goes down.
unknown (47:21):
Okay.
SPEAKER_02 (47:22):
Uh and are we getting better or worse at
handling these types ofincidents as an industry from
your experience?
Or about the same?
SPEAKER_01 (47:34):
I don't think that's a fair question because I don't
think I think we're having moreof them, but I don't believe
we're getting better.
It's not that we're gettingworse, but I don't think we're
getting we've been kind ofstatic about this.
We're we are static in how we dothings as an industry.
Yeah.
Um and then the incidents arehappening more often and maybe
(47:56):
even bigger impact.
So in a way, we're getting worsebecause we're not improving.
Okay.
Um, so that's probably how Iwould phrase that.
SPEAKER_02 (48:04):
And last question.
What is one question every CISOshould ask their SaaS vendors?
SPEAKER_01 (48:12):
How are you going to make sure that we can continue
to do business?
SPEAKER_02 (48:17):
It's kind of an important one.
Yeah.
I mean, because like we talkedabout, like when you're
outsourcing all of your securityto them, you know, they'll give
you the everything's redundant.
Yep, yep.
But maybe ask them straight up,what happens if Shiny Hunters
gets in and locks your stuffout?
What do we do?
Like, what do you do?
Well, I mean, they took it downthemselves.
(48:38):
Yeah.
The company took it down.
So well, yeah, but it's becausepeople were navigating to their
sources and getting pop-ups thatYeah, yeah.
I mean, did they see thenotification?
Yeah, they were Shiny Hunterswas they were roasting them,
unfortunately.
SPEAKER_01 (48:53):
As they should, yeah, as they should.
So no, uh that's not fair.
I I I said that wrong.
I don't want everybody.
No, no, no.
I think we ought to leave thisin because I think this is
really important because whenwe're talking from a security
perspective, yeah, we have adark humor about this.
Oh, for sure.
I think you have to, you haveto, yeah, but that comes across
in different ways to theaudience sometimes.
(49:16):
I don't want anybody attacked.
I don't.
And the fact that our customerswere down.
I have a lot of good people thatI work with on a regular,
including myself.
I don't sleep a lot anyway, butI didn't sleep a lot that that
night everything was down.
We were trying to make sure wehad plans for it to come up.
So I don't ever want people togo down, I don't want them to be
(49:38):
attacked.
But more importantly, I wantpeople to implement security
controls to make sure they'renot attacked.
And if someone gets into yoursystem, I get they're going to
make an announcement saying,hey, yeah, we're in.
Yeah.
Because some people, some peoplewill deny it.
So the obnoxiousness of thethreat actors can be helpful to
(50:02):
security people because they cansay, ah, we know they were here.
SPEAKER_02 (50:06):
You know, there's no doubt about that.
And and to kind of piggyback offthat, like as a in the sock, I
kind of look at it the same waywhere we uh I've talked about it
on the podcast that I do.
Like when I'm saying thisransomware group is out there
and they're killing it, or likeI've got respect for them, it
doesn't mean like I like whatthey do.
Yeah, you don't like whatthey're doing, but they got
(50:28):
skills.
But they they do it or they havesuch an approach in it that's
like it's it's kind of amazingthe way that they they they
change the face of what we know.
I mean, that's I mean, not tosay that we're always behind,
like our goal is to be ahead ofthem at all all times, but if
you don't know what the attackvectors are, you know, you're
(50:48):
just kind of identifying yourrisks and trying your best.
So um, yeah, I get that.
It's it's just uh um they'regood at what they do, you know.
Yeah, almost I won't I wouldn'tsay to like our benefit, but
like the way it changes, um itmakes us think differently as
(51:09):
well.
SPEAKER_01 (51:10):
Yeah, we have to think like the adapters.
Yeah, I mean, and and you'reright, it's a uh you give them
respect because I mean thesepeople are being successful.
Yeah, our job is to be just asgood.
Yeah and I'm just not justsaying us, you know, here at
alias, it's everybody whomanages the system.
(51:30):
If you have data of people oryour systems could hurt someone
if they go down, it's your jobto be as good as or better than
the attackers.
That's just the reality.
This is this is the job, this issecurity, you know, it's not
funsies, it's security.
(51:51):
Yep.
So these things are important.
We have to be better than theattackers, and when the
attackers best us, okay, I getit, let's get better.
SPEAKER_02 (52:00):
Yeah.
And not to harp on like get apen test again or anything.
Um but I mean, I see it with me,you know, we work with different
socks and stuff too.
We help socks that are they'regrowing and learning, we try to
help them harden uh their theirsecurity environment.
Um, and a lot of the times it'sjust it's just lack of
(52:21):
awareness.
Yep.
Um that coupled with like alertfatigue and there's nobody
really going in and um helpingto mitigate you know the type of
activity that's happening.
So when they when a real alertcomes in, it's like like that
person who received the emailwho didn't know what to do with
(52:41):
it, like uh we run into a lot ofthat.
So um I think just from a SOCperspective as well, we see this
third-party attack, and it's umyou vet your vendors, you know,
echoing everything that yousaid, look at your contracts,
um, and just from a security,just specifically like SOC team
(53:05):
perspective, um just be aware,like try to get a baseline of
like at least if hey, we seethis activity coming in, we
haven't seen it before.
Yep, look into it.
Yeah.
Um, I I've again we've run intoso many instances where you
we've got an organization that'slike, uh, we've seen something
similar before.
So we're just right, well, we'lljust we'll just monitor this.
(53:28):
And yeah.
Um again, the these guys shiftin their um their attack methods
so frequently.
Um you just gotta stay on top ofthe curve.
You know, you gotta be outthere, take up and read, you
know, like go look forinformation.
All right, so final closingthought, and then we can we can
(53:51):
be done.
Yeah, I think we're gonna gethooked here soon.
Oh, yeah, she's yeah, probably.
So I what is what would be Iguess just from a CISO
perspective, like what is theone if there's one thing you
would want organizations or a aa fellow CISO to take away from
(54:15):
this, from this incident, whatwould you hope that is?
SPEAKER_01 (54:20):
Even though it's not your fault, you still have to
clean up the mess.
SPEAKER_02 (54:23):
All right, that's fair.
And that is a wrap on thisepisode of the Secure AF
Podcast.
Uh, thank you so much forjoining us.
If you have questions about thisincident, want to talk to uh us
about CISO stuff or pen tests orwhatever, uh hit us up on our
social media or via our website.
Uh, as always, thanks for yourtime.
(54:45):
Bye.
SPEAKER_00 (54:54):
The Secure AF Podcast is a production of alias
Cybersecurity.
Visit us online ataliascybersecurity.com.
All rights reserved.
And I had a customer that reached out, one of our
customers reached out to me witha screenshot saying, um, hey, we
had a user navigate to thistraining website that we use,
and there's this shiny hunters,and we all take a screenshot and
send it to us.
And they were super concerned.
And this wasn't even a highered.
Yeah.
This was an organization thatjust utilized it for some
internal trainings.
(00:21):
Yeah.
Yeah.
And um that so that spawned likea whole like uh what's going on?
And it's like, hey, let's thisis a third party, you guys are
okay, this is what this means,kind of thing.
But it's kind of interestingthat they had no idea, and this
had been already, you know, inthe news for a while.
SPEAKER_00 (00:42):
You are now listening to the Secure AF
Podcast.
SPEAKER_02 (00:51):
Welcome to another episode of the Secure AF
Podcast.
Uh, I will be your host today,Andrew, uh security engineering
lead here at AliasCybersecurity.
And joining me today is mycolleague and mentor, uh,
Jonathan Kemet.
How are you doing, sir?
Mentor.
I like this.
Yeah, I was I would call youmentor for sure.
He's our CISO here at AliasCybersecurity.
SPEAKER_01 (01:11):
Uh I only teach good things sometimes.
SPEAKER_02 (01:14):
Usually, yeah.
Um, and today we're going to betalking about a subject that is
all over uh cybersecurityheadlines today for a good
reason.
And that is the Canvas uhplatform.
Would you would you call it adata breach?
Would we call it a breach atthis point?
SPEAKER_01 (01:32):
I I would say yeah, it's a breach.
Incidentally, in most cases, uhthe breach would require
notification, and this is goingto require notification.
Yep.
SPEAKER_02 (01:42):
Yeah, for sure.
That's fair.
Um, and this uh breach incident,breach, um, it affected a lot of
organizations.
Yes, a lot of organizations.
So uh it's just to start out,uh, for listeners who may not be
familiar with what exactly theCanvas platform is, um, who tip
(02:04):
who typically uses so Canvas isa learning management system.
SPEAKER_01 (02:09):
Um, so when you imagine someone going to school
and their instructors areputting up content for them up
online that they can go in andread or watch or submit uh
papers or submit stuff, uhthat's through the learning
management system.
And and generally speaking,there's three or four large
learning management systemsthroughout all of education.
(02:32):
Uh, Canvas is one of them.
It's a company calledInfrastructure and no
Instructure.
Um, I always get that messed up.
I always say Canvas, that's howeveryone calls it.
Uh, but the company is calledInstructure.
And uh it was it's it's verywell used.
I mean, and it's not just uhhigher ed or K through 12, a lot
(02:55):
of organizations use the sametool within their own training
programs, right?
Uh, and using it as a learningmanagement system.
That's what it's designed for.
Um, when you go on theirwebsite, you're gonna see some
really big names at the bottom.
There's a lot of people who useit.
SPEAKER_02 (03:09):
Yeah, yeah.
And I mean, my mother-in-law,who's a librarian at elementary
school here, you know, we wereover at their place yesterday
and she said, Okay, whathappened with Canvas?
Uh so I mean, a lot oforganizations, uh higher
education, elementary, like theywere down.
SPEAKER_01 (03:28):
Yeah.
It was hard down.
It was hard down for a littlewhile.
It's we've been having issueswith it for about a week all
last week.
Uh, but they took it completelyoffline Thursday afternoon and
it came back up in the middle ofthe night.
SPEAKER_02 (03:42):
Right.
So, how critical is Canvas forday-to-day operations for those
organizations like universities,enterprises, or just training
programs?
SPEAKER_01 (03:52):
Um, well, for universities in some K through
12s, they were hard down becauseum one of the things that
happened during COVID is schoolsystems started pushing all
their classes online.
And at that time, they chose thedifferent learning management
systems that they did.
Some did Blackboard, some didD2L, some did Canvas, you know,
(04:15):
whatever it was.
And so they they, in fact, Iknow some schools had a specific
mandate that every one of theirclasses would have
representation in the learningmanagement system.
Well, what that turned into isthat is how the class is run.
The that is how the students getin there, that's how the faculty
get in there and get theirstudent roster, that's how
(04:36):
students uh get their content,that's how they submit their
content.
It is really that's how wheretheir grades come from, their
grades get downloaded from theirthe the LMS into their student
information system.
I mean, it's critical.
Right.
Um, the the afternoon it wasdown, the hard down.
Uh there was universities thatliterally just shut down
(04:57):
classes.
They didn't, they couldn'tsubmit anything, they couldn't
do anything.
Now, one of the problems forthis particular incident and
this particular timing is it wasfinals.
You know, it was the last coupleof weeks of school.
This hit them hard.
They were trying to get thoselast grades in, and it was it
was very impactful.
SPEAKER_02 (05:16):
Right.
And that kind of launches meinto my last question here uh on
the setting the stage portion.
Um, from a CISO lens, how wouldyou classify Canvas, like
mission critical, sensitive, orboth?
SPEAKER_01 (05:30):
Oh, I I absolutely call it critical infrastructure.
Critical infrastructure for theorganizations that are using it
to that degree, it would beconsidered uh critical
infrastructure, right?
Critical systems.
Uh, because the primary role ofeducation, you know, K through
12 or higher ed is to provideacademic services to teach
(05:52):
students.
And if their their only way toteach is through that learning
management system and it's down,that's a problem.
Now, when I was going to school,we did have a version of
learning management.
Now, this was a very long timeago.
I got lots of gray.
We had a version of learningmanagement, but we only had a
handful of classes in it.
Right.
When I was at the university,when I was a CISO there, we
(06:16):
probably had a half to threequarters of our classes in it.
But again, after COVID, becauseof the what happened at COVID,
everyone had to have an onlinepresence.
Well, learning managementsystems was that online
presence, so they just movedeverything into it.
So now, I mean, there areinstitutions that don't have any
class that is not in the LMS tosome degree, and most of their
(06:40):
classes are entirely in the LMS.
That means their content, theydon't they don't even have
PowerPoint presentationsanymore, like outside of the the
the LMS.
So it's not like you're going toa class and they bring a
computer up and a PowerPoint.
It's they log in and all thatcontent is there.
So it is absolutely critical forthose institutions.
SPEAKER_02 (06:58):
Yeah, and that's another perfect transition into
talking about the dependency andlike risk surface here.
Oh, yeah.
So um you mentioned, you know,during COVID, there was a huge
shift um just trying to survivein most cases.
Yep.
Um, and that gets into thequestion of how heavily are
these organizations relying onon SaaS platforms like Canvas
(07:22):
compared to on-prem systemstoday.
SPEAKER_01 (07:25):
It's all SaaS.
I mean, most of theorganizations are moving to they
either have moved uh because ofCOVID andor other things, or
they are moving.
Um, and you know, when we thinkabout higher eds or even K
through 12s, and what we'retalking about is either
Microsoft or Google.
Yeah um they're running an LMSof some kind, their um their ERP
(07:49):
and student information systemis probably SaaS.
Some institutions still haveon-prem things, but a lot of the
organ or a lot of the providersare saying, we're not gonna
support that in the next twoyears.
You have to move off to our SaaSplatform, which makes sense from
that vendor perspective becausethey don't want to keep
maintaining old products, oldon-prem things.
(08:12):
So they're just saying, okay,we're not gonna do any more
updates to it.
You know, you don't have tomove, but you're not gonna be
using the service anymore.
So I would say mostorganizations probably have 70
to 90 percent in some version ofa SaaS.
Right.
Um for those that aren't,they're getting there.
(08:35):
I mean, they're they're they'reworking to that.
I would say in five years, uhprobably 90 to 95% is all gonna
be SaaS products.
SPEAKER_02 (08:43):
So what risks come with centralizing so much
operational and user data into asingle LMS platform?
SPEAKER_01 (08:53):
Well, the risks there's a couple different
risks.
Number one, you have no controlover it.
Right.
SPEAKER_02 (08:58):
You're outsourcing that security.
SPEAKER_01 (08:59):
Yeah, you you're everything is on that vendor.
So you've got to have the rightvendor management program, the
right contracts, the rightcheck-ins, all those different
things.
But then you're just prayingthat the vendor does good stuff.
Right.
I mean, we saw in thissituation, you know, we've had
9,000 institutions go down, orsomewhere around that 9,000.
(09:22):
There is nothing they could havedone.
There is absolutely, and I'vetried to get this across to the
institutions, there was nothingthey could have done any
different, unless we talk aboutbusiness continuity.
That's a little different.
But in terms of the the attack,the threat, the threat profile,
the vulnerabilities, that wasall on the vendor.
(09:44):
The vendor is the one thatmanages it.
So when you're looking at risk,that is high impact, hopefully
low probability.
So when you're doing your riskmatrix, the impact versus
probability, um, it's highimpact for sure, hopefully low
probability, uh, but it's notzero.
And in this particular case, itwas very clearly the vendor had
(10:07):
a problem, it had a huge impacton the institution.
Now, for higher eds, there'sonly a few things that really
have a negative impact shortterm.
You can affect higher ed longterm, but let's say you had the
internet go down, you know, atthe institution, probably a
couple of days before itactually caused harm of some uh
(10:30):
monetary harm or something else.
Um, but that's not the way withLMS when all of your teaching
capability is in that tool andit goes down, even just for us
in our tool, that is your entirebusiness.
So that one is huge, hugeimpact.
(10:51):
So you do a couple things.
I mean, you have to evaluatethat risk just like that.
What is the impact?
Huge.
What is the probability?
Probably pretty small, but notzero.
So you have to put in yourbusiness continuity in to make
sure that you take that intoaccount.
Hopefully, it never happens, butwe saw here it did.
So, and and I think there's alot of work, and this is as far
(11:12):
as I can remember, this is thefirst time that we've had a
major LMS outage like this.
I think so.
Um, and it no one was expectingit.
Yeah.
SPEAKER_02 (11:23):
So, in your experience, are organizations
generally overconfident in theseSaaS providers' security
controls?
SPEAKER_01 (11:33):
That's a good question.
I don't think they'reovercompetent.
I think they just don't know.
They don't know.
I mean, I you know, I I workwith organizations all the time
in their vendor managementprotocols, and they just they
don't ask questions, they don'thave an MSA in there as it
relates to uptime.
You know, you need to have anSLA, in some cases, OLAs, you've
got to have um, you know, whatis it that they're doing in
(11:55):
terms of their implementation ofsecurity controls?
What are they expecting risks tobe?
Now, you can't prepare foreverything.
I mean, this it's not evenreasonable to think that.
But some of the more commonthings you want to have an
alternative for.
So uh I can almost guarantee youthat from this point forward,
(12:15):
whenever someone is asking, youknow, what are your security
controls?
Another question they're gonnaask any LMS, or at least I hope
they do, is what happens if yoursystem goes down?
How do I get my data so I canfinish my semester?
How do I finish finals?
How do I do this?
What options are we going tohave?
And I'll tell you, no one has agood answer to that yet.
SPEAKER_02 (12:38):
All right, so let's talk about the attack itself.
All right.
Okay.
So at high level, uh, what do weknow so far about this canvas
breach?
SPEAKER_01 (12:50):
So, what we know is based off what we're being told.
Uh, so make sure that everyoneunderstands that.
So this is coming from Canvasand from what we have found out
on the internet for as much asthat's worth.
So we believe that the attackeris called Shiny Hunter, um, or
(13:11):
that is the attacker that didthe attack.
Uh, from the company, it wasthere's a tool called Free for
Teacher that Canvas provides.
And this is not their normallicensing structure for higher
rate institutions or K through12, it's something else, but
it's for a teacher that may nothave a big license, they could
(13:35):
get into this tool and use itand create for their classes.
Um, what we're being told fromthe company is there was a
problem within that tool thatallowed the attackers to get in
and then do effectively aprivesque or some version of
vulnerability to get into theback end systems of that tool.
(13:56):
So we don't know a lot ofdetails that hasn't been
released.
Uh, as of today, um, this wouldbe 5-11 of 2026.
I know that the uh person whodoes our podcast may not like me
saying that out loud, but thereality is it's important here.
As of today, we know theattackers got into that system
(14:17):
and the company is running anincident response investigation
to figure out what all hashappened.
And they said that they're goingto release at least parts of
that to the customer base.
So hopefully in the next coupleof days, we're going to have
more information about what theydo know and how that has
affected the rest of thesystems.
SPEAKER_02 (14:38):
Yeah, and I think right now they're saying that
the exposed data based onconfirmed statements or names,
email addresses, uh, studentIDs, and uh chats in chat.
So that would be internalmessages between Canvas users.
Yes, that's what they'rereporting.
SPEAKER_01 (14:56):
That's that is what they're reporting.
Um now I would say this that theDepartment of Education still
counts that information to beFERPA information.
So um there will always be aspin on this on any sort of
announcement out to the worldabout what is what is out there
(15:17):
and what you're going to need todo.
And you know, people will try todownplay as much as they can.
I mean, they're saying thingssuch as um there was no social
security numbers, there was nocredit card numbers, there was
no okay, great, but theinformation that was released,
um, the name, the the IDnumbers, the chats that can
still be considered FERPA data.
(15:37):
So there is still requirementsthat organizations are gonna
need to do uh breachnotifications, DOE
notifications, all kinds ofdifferent things.
So, again, this is what we'rebeing told.
Yeah, um, more information willcome out in the future after the
investigation is complete orafter they you know do as much
as they can.
(15:58):
Um, and I believe that the enddate for the threat actor is
tomorrow to release theinformation.
Now, one of the things that youknow we do this a lot in terms
of working with the attackersand working with the
organizations, you know, thatthe that middle piece there,
sometimes we don't knoweverything until they've
(16:21):
released the data.
And then sometimes we releasethey release the data and like,
oh, that was a lot more thanwhat we were expecting.
So or we were even seeingexactly.
So we don't know right now, wedon't know everything.
So as soon as the attackers, ifthey're going to release the
data, then we'll have at leastthat information to some extent
(16:43):
after they finish theinvestigation at the
organization level at the canvaslevel, we might have some more
information there.
We just don't know right now forsure.
SPEAKER_02 (16:52):
Gotcha.
So do we know if the motivationbehind this attack?
Like, was it financial?
Was it um espionage hacktivismor opportunistic?
SPEAKER_01 (17:04):
Um, I I think it was financial.
Um, I mean, I think that they'vecontacted them saying, hey, uh,
give us money, we're gonna getrid of your data.
Um I there has been reports thatindividual users have been
reached out to.
We don't know if these arescams, we don't know if it's
really the threat actor, butagain, it's a uh give us money
(17:25):
and we'll delete your data typething.
So I think at the moment, webelieve it is a financial
attack.
SPEAKER_02 (17:32):
Do we know how this breach was initially detected?
Like did the provider come out,or was this like the customers
were going, hey, we're canaccess our stuff.
We're seeing this pop-up.
What's happened?
SPEAKER_01 (17:44):
Um, so I've heard a couple of different things.
Some people there the the thethreat actor did post
information on websites, yeah,you know, that they had gotten
into it.
Um there was supposedlyinformation provided to them,
like a ransomware note or uhsome sort of communication
saying, Hey, we got in give usyour money.
(18:05):
Um, to my knowledge, at leastnone of my clients ever saw
anything from a usabilityperspective.
Nothing changed in theirversions or in their client or
in their uh in their webinterfaces.
They didn't see any of that.
They're going off what they werebeing told by the leadership by
from Canvas and by what they'reseeing in the news.
SPEAKER_02 (18:27):
Now I had a customer that reached out, one of our
customers reached out to me witha screenshot saying, uh, hey, we
had a user navigate to thistraining website that we use,
and there's this shiny hunters,and we'll take a screenshot and
send it to us.
And they were super concerned,and this wasn't even a higher
ed.
Yeah, this was an organizationthat just utilized it for some
(18:47):
internal trainings.
Yeah, yeah.
And um, that so that spawnedlike a whole like what's going
on?
And it's like, hey, let's thisis a third party, you guys are
okay, this is what this means,kind of thing.
But it's kind of interestingthat they had no idea, and this
had been already you know in thenews for a while.
Yeah.
SPEAKER_01 (19:07):
Um well, just like any other incident, people are
gonna try to downplay it becausethey don't want notoriety.
Yeah, um, and in this situation,the the tool itself is used so
widely, it's also wrapped up inother names.
So, I mean, they'll take thefunctionality and they'll put a
different name on it for thatorganization or brand it however
(19:29):
they need to.
So people may not have knownthat it was Canvas, right?
Because that was not how theycall it.
It's just an LMS to them.
Um, but most of my people that Italked to, their first notice
was a notice from Canvas thatsomething had happened, which
you know that came out, Ibelieve, on May 1st uh was the
(19:51):
first one that we saw.
And my talking about my client.
And uh there was three or fournotices after that.
Every every other day or so, wehad gotten notices with a little
bit more information about it.
But of course, we saw stuff onthe news, there's stuff from
other organizations, uh, stuffon X.
I mean, there's all kinds ofdifferent areas where people
(20:11):
were posting things, which canmake it difficult because you
don't ever know what is truthfuland what's not.
Even some of the information wewere getting from the
organization, it's not that theywere lying, they just didn't
know.
And so you you don't want toinfer anything or assume
anything.
You just you're trying to workwith the best information you
(20:32):
have at the time.
For sure.
SPEAKER_02 (20:34):
And so what does this incident tell us about uh
detecting blind spots in SaaSenvironments as an organization?
SPEAKER_01 (20:43):
Well, it's really hard.
It's really hard within the SaaSbecause you have to rely on the
SaaS to do that.
But the blind spots for me as aCISO is going to be business
continuity.
If this system is not available,um, how do we function?
SPEAKER_02 (20:58):
And I think going beyond just this event, that is
a common problem or common justtheme that we see doing as in a
response, where they have thistool that they utilize that
they've invested 100% of intozero backup strategy.
Like, no, like, yeah, hey, ifour one tool goes down or breaks
(21:23):
or disappears tomorrow, but butthat's not how it's sold.
SPEAKER_01 (21:28):
Right.
I mean, when you think about it,software as a service was sold
as we've got all these datacenters and we have all this
redundancy, we're always up.
Right.
I mean, think about AWS or Azureor anything, that that's how
it's sold is you know, you havefull redundancy, so we're always
up.
(21:49):
So we the the customers havebeen sold this, it's just and
and the salesperson may not haveever said it, right?
But the fact that you'rethinking as a That's how they're
thinking about it.
For sure.
So, and I'll and I'll tell you,Canvas was not down because of a
(22:10):
resource limitation.
Right.
It was not down because theirservers went down, it was down
because they turned it off.
They they went into amaintenance point, which don't
get me wrong, I think it was theright thing to do for them.
They were turning it off to getthe attacker out.
100% agree with that.
If I was there, that's probablywhat I would have done as well.
You know, and I don't want tosecond guess the CISO there.
(22:31):
You know, I'm sure that theywere all running, you know,
crazy just to get thingsfunctional.
But all we know is theyintentionally turned it off.
Now, going back to that riskthing, how do you know your your
vendor's not going to do thatfor you and you have no control
over it?
Right.
Now, even if they had an SLAsaying they won't do that, or
(22:56):
they'll have 99.999%, you know,five nines uptime, I mean, some
bad things happen, and the bestthing to do is turn it off.
Um you can't get around that.
And since you don't have anycontrol over it, all you what
you can do as the institution isokay, how do I continue
(23:17):
providing the functions?
Now, let's be real about this.
This was higher ed and K through12s, and this was learning.
This is academic stuff, and itwas very impactful to them.
For sure.
But it wasn't a hospital thatwas trying to provide patient
treatment, it wasn'tinfrastructure, you know, oil
(23:37):
and gas and electricity andwater and things like that.
That when it's down, you'reyou're talking minutes before
impact is felt.
So huge impact for sure, but itwasn't as bad as maybe another
industry they got here.
Or critical industry.
Right.
So, I mean, we got to balanceit.
(23:58):
Um, for my clients, you know,I'm gonna recommend, I am
recommending that we doimplement some better business
continuity.
How do we teach these things?
How do we still provide serviceseven if the SaaS goes down?
I'm not saying don't use it.
I'm just saying, well, let's,what are our alternatives?
Right.
You know, can we do somethingdifferent?
(24:21):
Maybe we have a backup of somecontent that, well, we'll just
get them online and we'll do itanyway.
You know, I I'll say this youknow, one of my clients had it
was a major impact.
Of course, it's the time ofyear.
And uh we were talking, it waslate one night that we were
still working on this, seven oreight, nine, ten o'clock,
whenever it was.
(24:41):
And one of the things I got themto understand is we don't make
any decisions right now.
You know, they had just turnedit off.
You know, Canvas had just turnedthe systems off.
Uh, let's say four o'clock,whenever it was.
I don't remember now.
And so we were planning, youknow, okay, what if it doesn't
come back on?
You know, what's worst casescenario?
(25:02):
It's not coming back on.
We have faculty members leaving,we have students trying to
graduate.
You know, what what is it we'regoing to have to do?
It's like, okay, I 100% agreewith thinking about this.
Don't make any changes.
Right.
Don't make any changes becausewe don't know yet.
And let's see what they do.
(25:23):
They're affecting 9,000institutions.
If they were going to be downfor a full day, they would
absolutely lose market share.
SPEAKER_02 (25:32):
Right.
SPEAKER_01 (25:33):
So everything came back on about 11 or 12, is what
we think.
Uh, we got notices at aboutthree in the morning saying
everything was back up.
And they were able to go throughthe next day fully functional.
Now we didn't know much, but weknew that the tool was
functional and were able tofinish out the semester, finish
out that week.
(25:54):
Um I do appreciate them notbeing down long, even though I
understand why they were down,but it allowed the institution
to be able to finish out thelast little pieces that they
needed.
SPEAKER_02 (26:08):
Yeah.
So we've talked about theinitial impact to organizations,
but what kind of downstreamrisks are we going to see from
this?
Um, and I'm thinking about, youknow, based on what we know,
what data may have left, uh, orpossibly more, kind of depending
on what they gain access to.
Uh, but down downstream risk,kind of thinking about like
(26:31):
credential reuse, um, any kindof future phishing campaigns,
yep.
Uh, or uh identity theft.
SPEAKER_01 (26:37):
Like, I I think all of those are absolutely going to
happen.
You know, I think students, thestudents' names and their emails
are going to are out there now.
Um, I think attackers or maybecopycat attackers are going to
say, hey, give us X amount ofmoney and we'll delete your
data.
You know, they might say We'veseen that in the past.
Absolutely, we have.
Um, so I think that that isthat's going to happen.
(26:58):
Um, I think one of the thingsthat no one is really talking
about right now, but when ShinyHunter did this, this is huge
impact for higher ed.
I think other LMSs are going tobe targeted much more heavily
now because now they saw whatthe impact was.
So I think they're going to goafter them.
(27:19):
So if I was an LMS group, Iwould be absolutely learning
from this.
And I would be making sure thatthere you've got segmentation,
you've got MFA, you've gotanything and everything you can
think of to deploy in yourenvironment to prevent something
like this, and have strategiesfor if it were to happen.
(27:39):
So I think that will absolutelyhappen.
Um, yeah, maybe have some pentesters come in.
Yes, pen testers and riskassessments now it's and
contract management.
I some of these contracts arepretty bad and they don't have
SLAs in them, they don't haveany of these things.
So I think that those are goingto be, you know, on the uptick.
The other thing that we saw fromanother data breach from about a
(28:02):
year and a half ago wasinformation that was gathered in
that data breach was used tocreate fake accounts in the
higher ed, which was then usedto get federal financial aid.
So it was uh financial aidfraud, but it was real students.
It was the real studentinformation.
So I think that as that gets outthere, you might see an uptick
(28:23):
in fraudulent applications intoinstitutions trying to get in to
get a user account to be usedfor other things, or to even
apply for federal financial aid.
SPEAKER_02 (28:31):
Absolutely.
Yeah, I didn't think about thatone.
So that's a really good pointthere.
So let's get into your favoritetopic of this.
What kind of regulatory orcompliance issues are these
affected organizations going tobe looking at?
Well, they have to report.
SPEAKER_01 (28:50):
I mean, no matter what this is, even though this
is a third-party vendor, you area controller.
So a higher institution, eventhough this is your vendor that
got compromised, you're thecontroller of the data.
You have to do the reporting.
You have to report it to theregulatory side, you're gonna
have to report it to the usersas a hitch breach notification
laws, all the state laws andeven country laws, meaning if
(29:12):
you have an internationalstudent base, you still have to
report this.
Now, the reporting is prettysimple.
Hey, our vendor got hit.
Now, the DOE absolutely knowsabout this.
They are still going to want youto report because it's your
responsibility to report thisappropriately.
You will have to do the briefnotification, so you're gonna
have to let your students know.
(29:33):
And you have to say, it wasn'tus, it was our vendor, but we're
letting you know what thishappened because there still has
to be a risk assessment done atthat student level.
The student has to look at itand go, okay, well, maybe I am
going to see a lot more phishingattempts.
Maybe I am going to see thisstuff, maybe you know, password
reuse, whatever.
So they're gonna have to do arisk assessment of themselves if
(29:54):
they know it happened.
Now, I think everyone prettymuch knows it happened, but you
have to do those officialnotifications.
You have to document it, youhave to get it out to them, you
have to you use your bestprocess to get that out there.
Um, you also will have to doremediation processes.
How do you prevent this in thefuture?
Now, this is vendor management.
(30:15):
It's the vendor that had theproblem, but you have to do
vendor management processes tohelp mitigate this problem in
the future.
So I would absolutely be usingthis as a tabletop example, as
some process that you have aplaybook for of what happens
when it happens again.
What did we learn from it?
How do we resolve this?
You know, what is the timing?
(30:36):
Maybe you have to if you have afour-hour response time from
your vendor to let you know thatthere's an issue, well, then you
can have a six hour before yousend out a note to your campus.
So, what does your playbook sayabout this?
This happened.
Now, I put stuff like this on mytabletops all the time, and
everyone's like, Oh, that willnever happen.
(30:57):
It did happen.
Yeah.
So let's build a playbookbecause it will probably happen
again.
SPEAKER_02 (31:02):
That's funny to me on the the several tabletops
that I've been with you, which Ilove doing on.
Uh, I love your hey, you're nothere, or you got hit by a train
coming in and you're not here.
But how many times have we beenon an incident and the guy we
need to talk to is on vacation?
Arupa.
SPEAKER_01 (31:19):
Yeah.
Well, we just had one where thethe uh a stakeholder that we
needed to talk to was on acruise ship.
Yeah.
It's like, okay, well, that'sout.
Yeah.
SPEAKER_02 (31:28):
Yeah.
So you just, you know, you gottahave backup plans, you gotta
have your chain of command umfor sure.
So we've kind of talked aboutlike where the responsibility
falls between uh the platformprovider and the customer.
So, what should organizationsrealistically expect from their
SaaS providers uh just in termsof security and transparency?
SPEAKER_01 (31:51):
Well, okay.
I'm gonna say it a very specificway, they should expect what the
contract says.
Right.
Now, you could have elevatedexpectations depending on if
it's a critical function or not,but whatever that expectation
is, it needs to be in thecontract.
So if it's a certain amount oftime before they report it to
(32:11):
you or they report to youcertain information, whatever it
is, you want to make sure thatis in the contract.
Now, for higher ed, uh, and evensome for uh K through 12, uh,
you have a 24-hour requirementbefore you have to report things
to the Department of Ed and theState Department of Ed and all
kinds of different things.
So 24 hours.
So you really need to knowsomething of a suspected
(32:35):
incident within that 24 hours.
That needs to be in thecontract.
One of the other things that wefound out is we one of my
clients, we didn't know who thenotifications were going to.
Right.
So the organization, Canvas inthis case or instructor, was
sending out notifications.
(32:55):
We didn't know who was supposedto receive those.
Now, sometimes it's the personwho signed it, sometimes it's a
specific person, but is thatspecific person still there?
So those sorts of things comeinto play.
So you need to know what yourcontract says, and then when a
situation happens like this,read your contract because that
(33:16):
contract is going to tell youwhat to expect, when to expect
it, and if you don't like it,get a new contract.
Right.
When it renews, you make surethat stuff is in there.
Contract management is huge, andI I harp on this a lot because
we have been in incidents whereI asked to see the contract, and
they're like, Oh, we don't haveit.
Then you have no expectation ofanything at that point.
(33:39):
You need to put thoseexpectations in before you need
them.
So that's what I would say isyou're going to get hopefully
what the contract says.
Sometimes not.
Sometimes they will not give youthat information.
Because here's kind of a redflag there, though.
Well, but I mean, we have seenit ourselves where people are so
(34:01):
tied up, and you know, you'reyou you've got a system scene.
It started on May 1st in termsof the first notification.
I'm sure their teams wereworking 24-7 for six or seven
days straight.
Right.
They're not thinking about thecommunication side, they're just
trying to get it figured out.
So sometimes they don't.
I I I've got one story, Iusually do a lot of stories, but
(34:24):
this is the one story.
We had a data breach of a SaaS,and we were notified like three
months later.
Woof.
Um, it went to someone thatdidn't really know what to do
with it, so they ignored theemail in terms of coming into
the organization.
An IT person just literallyhappened to see it on the screen
(34:48):
when he remoted into hercomputer and says, That's odd.
Why what is that email?
So that finally got to me.
I'm reading it and going, Ohgod, what what no one told us
about this?
Well, skip ahead.
Um, three weeks.
We were very early in terms ofcommunicating with the that
(35:10):
team, and I was able to actuallyget a meeting with their CISO.
Now, this would be like gettinga meeting with the CISO at
Canvas, right?
Really difficult, but I I wasjust fortunate.
I had lots of lawyers on ourside, lots of lawyers on our
side their side, but I got totalk to him.
So we got on a phone call.
I had about 10 minutes, I askedhim a bunch of questions.
(35:33):
He actually did a good job.
Yeah, everything he did wasright.
I I was like, okay, I appreciatethat.
He did what was right, Iunderstand why he did, why he
did it.
It was fine.
Well, he got off, and then theCEO for that division got on,
and the question, the finalquestion from the our lawyers
(35:57):
was okay, the contract, the thecontract says you will tell us
within three days.
It took you like 60 or 90 daysor whatever it was to notify us.
Why that is a completelylegitimate question.
Yeah, contract says this, youdidn't do this.
And his response, now I'm I'moff at this point.
(36:21):
I've my my camera's off, my micis off, I'm done.
And he goes, Well, I'm reallyglad you asked that question
because I've been doing myresearch and it doesn't look
like anybody ever meets thoserequirements in that contract.
And then he turns to me and hegoes, Well, Jonathan, what do
you think about that?
(36:44):
I I turn my camera on and Iturned on my uh my mic and go,
Well, I'm really glad you askedthat because I teach a contracts
course for cybersecurityprofessionals at various
conferences.
And so my question back to youis if we don't have to follow
this piece of the contract, dowe have to follow any of the
(37:04):
contract?
Right.
They didn't like that question.
Of course not.
And their lawyers cut the call.
Just nope, fuck.
So the contract is reallyimportant, and I harp on this.
I mean, people make fun of mebecause I like reading
contracts.
Well, the reason I like readingcontracts is I don't want to
(37:24):
ever get caught off guard.
I want to know exactly what isin that contract, what we have
to do, what we're supposed todo, what we don't have to do.
Your contract, your masterservice agreement, whatever you
call it, is critical.
You spend the time up front sothat when you have problems
later, you have a contractthat's going to back you.
Yeah.
SPEAKER_02 (37:44):
It's part of the having a plan.
Exactly.
Exactly.
So we've discussed a lot aboutlessons learned already just
through this conversation.
But what role would like loggingand monitoring play here,
especially when visibility islimited?
SPEAKER_01 (38:02):
Um, so for my clients, what I told them in the
very beginning is review yourlogging to see if because we
didn't know anything, see ifanything was coming back.
Anything was affecting oursystems that looks odd.
Anything in terms of the datacoming from Canvas back to us
looks odd.
(38:22):
So we were trying, I mean, yourlogging is you is you when you
don't know what's going on, yourlogging is key.
I mean, that's how you deal withthis.
So um, in the very first day,first hours, we were absolutely
looking at that logging.
Yeah.
Um, we couldn't tell anythingbecause it wasn't that sort of
an attack, but we didn't knowthat.
All we know is that there was aproblem.
(38:44):
Our number one priority is toprotect our systems.
And that's where we went in.
We started looking at the logsof our systems to see if any
problems, any user accounts, anydata, anything looked odd so
that we could deal with it.
SPEAKER_02 (38:58):
Right.
So let's let's talk about somepractical recommendations for
security.
Okay.
Some of them we've alreadytouched on, um, like talking
about like what theseorganizations could do or what
steps they would take if uh theyuse Canvas or similar platforms.
But how could an organizationstrengthen their incident
(39:20):
response plans for third-partybreaches like this?
SPEAKER_01 (39:24):
Well, first they need to realize that it is the
risk.
I mean, what risks there are.
So you need to plan that in, youhave a risk assessment around
that.
You need to, of course, havethat the contract side of what's
going to happen during, youknow, the during the situation
where how they're going tocommunicate.
Um, but it really does come downto business continuity.
I mean, you need to understandthat if the risk, here's the
(39:48):
risk.
A lot of people will look at therisk of the data got
compromised, and that's bad, andwe have to deal with that.
But the other risk is youcouldn't teach classes, right?
You couldn't graduate, youcouldn't do this.
One of the things I asked one ofour clients was, What did when
you can't do a final, how do youdeal with that?
(40:10):
Do you postpone it till nextweek?
Do you just say, Well, whatevergrade you get is what you got,
right?
Um, what is it?
And they were like, We've nevermissed a final.
I'm like, I know we missedfinals during COVID.
SPEAKER_02 (40:22):
So but my question is, these or I mean, Canvas
hasn't been around forever,right?
These organizations had beendoing I mean LMS's have been
around for probably 20 years.
Yeah, I know, but before I meanschools have been around longer
than 20 years.
Well, well, just what were theybefore?
You know, this is this gets intolike my like I don't know why I
(40:44):
think just having like the thesimplest backup plan is the most
effective.
Like what how did we do it 20years ago before the LMS?
SPEAKER_01 (40:54):
Well, but but here's the difference.
I mean, I understand the backupplan, but schools can't be
redundant, they don't have theresources to be redundant.
So if you have all of yourcontent in your LMS, you're not
gonna maintain your contentsomewhere else.
That's redundant, right?
That's that's more effort, morebackup, more whatever.
The whole intent is to get it towhere the the tool itself is the
(41:17):
redundancy.
In this particular case, theytook that down.
So I get it.
Um, and one of my questions waswell, why can't you just do a
Zoom call and teach anyway?
Yeah, I mean, you have thecontent, you should know what
the syllabus should say, whatthe content is.
Well, yeah, the syllabus was inthe LMS.
They didn't have the syllabus,they didn't know what content.
Now, this was finals, so thisthey weren't actually teaching
(41:38):
like that.
They were trying to get theirfiles in and get their grades
out of there.
Uh, but I will say I had oneclient who uh they still
maintain gradebooks.
Yeah, yeah.
If for those of us who are old,the old grade books that were
green and they had the lines inthere, they still maintain
those.
So they had all the grades.
Now we had a way to get thosegrades into this the student
(42:00):
information system, um, but veryfew people have gradebooks
anymore.
Yeah.
So it that's the thing is you'vegot to have the backups,
absolutely, but they don't havethe resources to be redundant,
so a lot of them don't do that.
Okay.
SPEAKER_02 (42:14):
Let's get into executive and board level
perspectives.
Okay.
Because I know you're gonna havesome great answers here.
Because these are theconversations I don't want to be
in.
Yeah.
So, how should CISOs communicateincidents like this to
executives and the boards?
SPEAKER_01 (42:31):
If this is the first time your board is hearing about
your issue, yeah, let's say thisis the very first time.
Yeah, this is this is a problem.
Um your board, when you'retalking to your board, they
should not, you should not bethey should not be surprised
that a vendor going down hascaused such an impact to the
(42:54):
organization.
That should be in your riskregister, right?
People should know, oh, theseare critical systems.
If they go down, criticalfunctions fail.
So uh if you've never done thatbefore, uh, you're gonna take a
hit on the chin.
Not because they went down, butbecause people were not aware of
(43:14):
it.
So communication here is key.
You need to get in early, youneed to explain to the board
here are all the systems that,if they go down, have a negative
impact on us being able tofunction.
They need to be aware of that.
Now they're gonna ask you, well,what are you doing to mitigate
that risk?
And some things you can do andsome things you can't do, but
(43:36):
that's a risk appetite issue ofwhat will the board allow you to
do?
Now, the board says, Nope, youhave to have full redundancy of
your LMS.
Okay, so we're gonna have twoone is Blackboard, one is
Canvas, right?
And you're gonna load your yourclasses in both, and people can
go to both, and that's really,really difficult in higher ed.
(43:58):
Absolutely.
So
SPEAKER_02 (44:00):
Then you have people who are like, I want to go do
this and it's not here.
Exactly.
It's like you can't hold meaccountable for that.
SPEAKER_01 (44:06):
Right.
Your system is messed up.
But I think it is fair to let Imean, you have to let the board
know that this is a risk.
Right.
Now, if they know it's a riskand it happens, what you're
doing is you're going to theboard quickly and going, Okay,
here's where we're at.
This is what we're communicatedwith.
This is what our expectationsare based off the contract.
Here's where we're at in termsof notifications and
(44:27):
communications with legal, withregulatory, with whatever.
You have your playbook ready togo.
You should have a playbook forevery single critical function
you do.
That if it's not there, what areyou doing to keep the business
running?
Right.
Now, if you have that playbook,you're walking in with that
playbook.
You walk in there, you go,here's what we know.
Here's everything that they'vesent us.
(44:49):
This is what we've seen in ours.
Here's what we're doing nextwith the playbook.
I think that is the mostcritical piece that any system,
any security professional or ACso when they walk into that
board, they need to have thatstuff ready to go.
If you just walk in gettingready to take a hit, it's like,
oh, well, I don't know anything,but I'll go in there.
Well, yeah, you're gonna take ahit.
(45:10):
And you're probably not gonna beasked back because if you're not
providing value to the board,right, they're not gonna have
anything.
Right.
So that's what I would say isyou should prepare the board
beforehand, that these arecritical functions.
I'm talking months, years ahead.
They should be completely awareof what your critical functions
are.
And if Canvas goes offline, thisis the problem.
(45:33):
That way they know about it.
And they're like, okay, got it.
We understand that's critical.
We understand we can't doanything about that.
Well, we will continue on.
SPEAKER_02 (45:42):
Yeah, it just gets into understanding the risks.
SPEAKER_01 (45:45):
Exactly.
SPEAKER_02 (45:46):
Exactly.
Have there been any risk framingthat resonated most when
discussing these third-party uhbreaches with your clients?
SPEAKER_01 (45:58):
Of the two that I I always go after is the
regulatory side and the impactto the users.
Right.
You know, what what is going tobe the impact to the students in
this?
Not from the fact they couldn'tget their grades in.
What is going to be the impactof that data being lost?
You know, I I'm I will say it,you know, every single
presentation, our job is toprotect people.
(46:19):
So I always make sure that whenwe have a data breach, I make
the leadership completely awareof what risks we have to the
organization for sure.
But also what are the risks tothe customers or the students or
the the patients, whatever?
Because I I think that is reallyimportant for us to understand.
For sure.
SPEAKER_02 (46:38):
All right.
So let's get into some closingthoughts now.
Okay.
Uh couple questions.
One, what's one thing everysecurity leader or Cisho CISO
that should what should theytake away from this incident?
SPEAKER_01 (46:54):
This is a vendor.
I mean, this is what happens.
You put that much trust in avendor, a vendor can absolutely
take down your business.
SPEAKER_02 (47:02):
What should an organization be doing today to
prepare for the next SaaSbreach?
SPEAKER_01 (47:07):
You need to understand all of your SaaS,
what are critical functions, andthen what is your business
continuity around thosebusiness, those business
functions, those criticalfunctions that will make sure
you can continue to work even ifthis goes down.
unknown (47:21):
Okay.
SPEAKER_02 (47:22):
Uh and are we getting better or worse at
handling these types ofincidents as an industry from
your experience?
Or about the same?
SPEAKER_01 (47:34):
I don't think that's a fair question because I don't
think I think we're having moreof them, but I don't believe
we're getting better.
It's not that we're gettingworse, but I don't think we're
getting we've been kind ofstatic about this.
We're we are static in how we dothings as an industry.
Yeah.
Um and then the incidents arehappening more often and maybe
(47:56):
even bigger impact.
So in a way, we're getting worsebecause we're not improving.
Okay.
Um, so that's probably how Iwould phrase that.
SPEAKER_02 (48:04):
And last question.
What is one question every CISOshould ask their SaaS vendors?
SPEAKER_01 (48:12):
How are you going to make sure that we can continue
to do business?
SPEAKER_02 (48:17):
It's kind of an important one.
Yeah.
I mean, because like we talkedabout, like when you're
outsourcing all of your securityto them, you know, they'll give
you the everything's redundant.
Yep, yep.
But maybe ask them straight up,what happens if Shiny Hunters
gets in and locks your stuffout?
What do we do?
Like, what do you do?
Well, I mean, they took it downthemselves.
(48:38):
Yeah.
The company took it down.
So well, yeah, but it's becausepeople were navigating to their
sources and getting pop-ups thatYeah, yeah.
I mean, did they see thenotification?
Yeah, they were Shiny Hunterswas they were roasting them,
unfortunately.
SPEAKER_01 (48:53):
As they should, yeah, as they should.
So no, uh that's not fair.
I I I said that wrong.
I don't want everybody.
No, no, no.
I think we ought to leave thisin because I think this is
really important because whenwe're talking from a security
perspective, yeah, we have adark humor about this.
Oh, for sure.
I think you have to, you haveto, yeah, but that comes across
in different ways to theaudience sometimes.
(49:16):
I don't want anybody attacked.
I don't.
And the fact that our customerswere down.
I have a lot of good people thatI work with on a regular,
including myself.
I don't sleep a lot anyway, butI didn't sleep a lot that that
night everything was down.
We were trying to make sure wehad plans for it to come up.
So I don't ever want people togo down, I don't want them to be
(49:38):
attacked.
But more importantly, I wantpeople to implement security
controls to make sure they'renot attacked.
And if someone gets into yoursystem, I get they're going to
make an announcement saying,hey, yeah, we're in.
Yeah.
Because some people, some peoplewill deny it.
So the obnoxiousness of thethreat actors can be helpful to
(50:02):
security people because they cansay, ah, we know they were here.
SPEAKER_02 (50:06):
You know, there's no doubt about that.
And and to kind of piggyback offthat, like as a in the sock, I
kind of look at it the same waywhere we uh I've talked about it
on the podcast that I do.
Like when I'm saying thisransomware group is out there
and they're killing it, or likeI've got respect for them, it
doesn't mean like I like whatthey do.
Yeah, you don't like whatthey're doing, but they got
(50:28):
skills.
But they they do it or they havesuch an approach in it that's
like it's it's kind of amazingthe way that they they they
change the face of what we know.
I mean, that's I mean, not tosay that we're always behind,
like our goal is to be ahead ofthem at all all times, but if
you don't know what the attackvectors are, you know, you're
(50:48):
just kind of identifying yourrisks and trying your best.
So um, yeah, I get that.
It's it's just uh um they'regood at what they do, you know.
Yeah, almost I won't I wouldn'tsay to like our benefit, but
like the way it changes, um itmakes us think differently as
(51:09):
well.
SPEAKER_01 (51:10):
Yeah, we have to think like the adapters.
Yeah, I mean, and and you'reright, it's a uh you give them
respect because I mean thesepeople are being successful.
Yeah, our job is to be just asgood.
Yeah and I'm just not justsaying us, you know, here at
alias, it's everybody whomanages the system.
(51:30):
If you have data of people oryour systems could hurt someone
if they go down, it's your jobto be as good as or better than
the attackers.
That's just the reality.
This is this is the job, this issecurity, you know, it's not
funsies, it's security.
(51:51):
Yep.
So these things are important.
We have to be better than theattackers, and when the
attackers best us, okay, I getit, let's get better.
SPEAKER_02 (52:00):
Yeah.
And not to harp on like get apen test again or anything.
Um but I mean, I see it with me,you know, we work with different
socks and stuff too.
We help socks that are they'regrowing and learning, we try to
help them harden uh their theirsecurity environment.
Um, and a lot of the times it'sjust it's just lack of
(52:21):
awareness.
Yep.
Um that coupled with like alertfatigue and there's nobody
really going in and um helpingto mitigate you know the type of
activity that's happening.
So when they when a real alertcomes in, it's like like that
person who received the emailwho didn't know what to do with
(52:41):
it, like uh we run into a lot ofthat.
So um I think just from a SOCperspective as well, we see this
third-party attack, and it's umyou vet your vendors, you know,
echoing everything that yousaid, look at your contracts,
um, and just from a security,just specifically like SOC team
(53:05):
perspective, um just be aware,like try to get a baseline of
like at least if hey, we seethis activity coming in, we
haven't seen it before.
Yep, look into it.
Yeah.
Um, I I've again we've run intoso many instances where you
we've got an organization that'slike, uh, we've seen something
similar before.
So we're just right, well, we'lljust we'll just monitor this.
(53:28):
And yeah.
Um again, the these guys shiftin their um their attack methods
so frequently.
Um you just gotta stay on top ofthe curve.
You know, you gotta be outthere, take up and read, you
know, like go look forinformation.
All right, so final closingthought, and then we can we can
(53:51):
be done.
Yeah, I think we're gonna gethooked here soon.
Oh, yeah, she's yeah, probably.
So I what is what would be Iguess just from a CISO
perspective, like what is theone if there's one thing you
would want organizations or a aa fellow CISO to take away from
(54:15):
this, from this incident, whatwould you hope that is?
SPEAKER_01 (54:20):
Even though it's not your fault, you still have to
clean up the mess.
SPEAKER_02 (54:23):
All right, that's fair.
And that is a wrap on thisepisode of the Secure AF
Podcast.
Uh, thank you so much forjoining us.
If you have questions about thisincident, want to talk to uh us
about CISO stuff or pen tests orwhatever, uh hit us up on our
social media or via our website.
Uh, as always, thanks for yourtime.
(54:45):
Bye.
SPEAKER_00 (54:54):
The Secure AF Podcast is a production of alias
Cybersecurity.
Visit us online ataliascybersecurity.com.
All rights reserved.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Bobby Bones Show
Listen to 'The Bobby Bones Show' by downloading the daily full replay.