November 19, 2025 • 46 mins
This story has a happy ending. In January 2024, the “not if but when” happened to us. We’d been cyber-attacked, and the bad guys were in the house.
Presenters:
Luke Allpress, CETL, MEd
Director of Innovative Solutions
Agua Fria Union High School District
Avondale, AZ
Brandon Gabel
Director of Information Technology
Agua Fria Union High School District
Avondale, AZ
It started off with a fairly normal outage. Internal sites were stuck, and printing stopped working. We halted all network traffic as we methodically checked possibilities: power, hardware, network servers/services. Then, our Manager of Network and Security found it—a service account doing way more than it should.
He immediately initiated a quarantine, isolating all network traffic, “locking the bad guys in the house” as we began our investigation. The two aims of a cyber threat actor, 1) exfiltrate data, 2) lock us out for ransom, were both mitigated by his quick, informed action. We survived with little loss to operations and no data loss, thanks to our plan. Come hear about the preparation and lessons learned from our first cyber incident.
Key Take Aways:
- We shape our workshops through Adult Learning Theory, emphasizing the expertise participants bring to the session and making ample space to apply new knowledge to existing problems. They will be reflecting on their own security plans and applying our lessons to their own situations.
- Have a plan (CIRP). The worst time to figure out your cyber security plan is the day you need one. Call your network and insurance partners NOW to discuss your CIRP, not when it happens.
- The guts: Know your backups and the backup solution/plan, make sure you’re backing up all servers. Audit admin access regularly. Ensure all accounts and devices, that can be, are locked behind MFA.
Slides, templates, etc.
Watch the webinar:
https://www.youtube.com/watch?v=REzmsuKmIkw
The Sessions Everyone Was Talking About Webinar Series
Missed CoSN2025 in Seattle or couldn’t attend every session? Don’t worry—we’re bringing the most popular, standing-room-only presentations to you in a special webinar series. Learn from top EdTech leaders from across the country—no travel needed!
CoSN is vendor neutral and does not endorse products or services. Any mention of a specific solution is for contextual purposes.
For a complete listing of all CoSN's webinars, please visit:
https://www.cosn.org/
Produced in partnership with edCircuit.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:07):
All right, let's kick it off everyone.
My name is Luke Alpress and and this is Brandon.
We will introduce ourselves morein a little bit, but thanks for
attending or reviewing this Coast in webinar series on
follow up from Coast in 2025. So this is a session that we
gave back in the Seattle Coast in conference and we're excited
to share it again with you in this webinar focus with webinar
(00:28):
medium. Hopefully they can just give an
idea of what it went it was likefor us to go through.
Yeah, cyber incident, introduce ourselves really briefly.
One of us is a former teacher and acedo and like overseas,
like data privacy and presentation.
One of us is an Army vet who fights off the bad guys.
And I don't know if you can see the two of us already, but you
(00:49):
might be able to guess which is which.
I taught math and physics and all those things for a while and
now I, I, I work in the IT department, but adjacent to it,
just helping data provisioning and data engineering analysis,
things like that. And over to Brandon.
Yeah, I'm the the Army vet. Really, I just wanted to blow
stuff up when I was a high schooler, but then I ended up
(01:11):
doing computer stuff in the Armyand loved it kind of.
And here I am. I just recently finished the
masters in IT management. I've been working in this
district for almost five years now, started in 2021 as a site
technician and just kind of keptworking my way up and that
little line at the bottom right there thrice and chaos.
(01:32):
I was in my prime when when we're going through this,
although it sucked, I I really wish we didn't have to go
through it, but yeah, yeah. I'm glad Brandon's on our side.
This is like an overview of QR code and a bit late to access
any of the resources we reference.
There's like emails that we sentout, like this is what we sent
to the community. This is a reference to our CRP.
(01:54):
All that can be found at those links.
Those are e-mail addresses. This slide that can be found
there as well. And also after the Cosin events
in April, there was a feature done on Brandon in this
incident. And so there's a link to that
article there. Brandon, I didn't tell you I put
that on that slide, but it's toogood of a picture of Marcio.
OK, So when we were doing this live, we threw a mentometer up,
but we will show this is the results we got from the
(02:17):
mentometer because we wanted to see who was in the room like
what. Let's calibrate this
presentation to experience that we have coming in.
And so we wanted to look like maybe the difference between the
individuals in the room that, you know, the CTOS, the cyber
experts who are really excited to hear about a cyber incident
and maybe our perceptions of thereadiness of our educational
(02:40):
institutions as a whole. So personal cyber knowledge
people maybe humbly didn't like fully rate themselves all the
way up at 5 stars, but you can see a pretty good nice hump
towards the higher end of things.
Like, Yep, I, I think I know what I'm talking about.
I think our personal infrastructure is good.
Then we get over to like, all right, how's the rest of our
leadership? Do we trust, Do we trust our
(03:00):
staff to not click a phishing link?
You know, and that one definitely got further towards
the less prepared side. And then the very the bottom one
was a fascinating like a very flat like do we have a CIRP full
like that was a across the boardanyone my land anywhere on that.
And so that was a big part of our story.
So Brandon will definitely get into that later.
(03:22):
Less fun to do this in a webinarformat, but we don't we are all
we all share the same woes of like turning to something off
and on again is the right solution.
Has someone sent you the password in the body of an
e-mail? Right.
We have all lived in this IT world and the Internet is broken
only to find out something is unplugged.
(03:44):
OK quick introduction just to set the stage.
I would imagine a lot of the folks that are interested in
attending this webinar will knowsome of this intro content.
But just to at least do some level setting and get us to the
place of having some common language and just getting us all
in the mindset. Education is the second most
impacted industry from ransomware attacks.
Ransomware being the bad guys come in, they try to lock you
(04:06):
out and say, pay us with Bitcoinor something comparable to get
your data back. And government education and
healthcare are targeted because we have lots of data that we
care a lot about and we don't necessarily have enough money
just flowing around everywhere to spend on a really strong
cybersecurity infrastructure. So it makes us susceptible to
that. Today we're going to talk about,
(04:28):
I mean obviously the the title of the session is we there was a
cyber instant aid attempted cyber attack and I always want
to be really careful with this language.
We'll talk about what went down there, what we did to response,
what we learned. And a big part of it is how we
were prepared for it because Brandon did a lot of things that
made us come to this successfully because of the pre
(04:49):
work he had done there. I will give a brief layman's
analogy of what happens in an attempted ransomware attack from
our perspective. So if our data warehouse, if our
on site data can be viewed as a house, the bad guys find some
way to get in. They find a spare key that was
(05:10):
left under the mat. You know, your, your kid tells
the the neighbor down the streetwhere they can find a spare key
and what the code to the security system is just because
they thought it was their friend, You know, so the bad
guys get in and then they can start going maybe into different
rooms of the house. And hopefully we have a really
good security system that says, hey, someone has entered the
(05:30):
house with suspicious behavior of some sort.
Hey, someone has entered this room of the house being a
different server with suspiciousbehavior.
And then ideally the security walls come down.
They prevent them from leaving, but they might still be in the
house. So then we still got to monitor
what they're doing, but then we get do get to track and see if
anything has left the house because we have monitoring on
all of the doors and windows to see what goes in and out.
(05:53):
But the goal being that we prevent them from taking
anything out of the house that would be exfiltration.
And then also in a ransomware attack specifically they're
gonna go in there and try to change the locks on all the
doors and say we will give you the new key if you pay us for
it. So just with that kind of as our
layman's analogy metaphor, I will toss it over to Brandon who
(06:13):
will give more of the technical details that I would imagine
most people watching us are looking for.
All right, So what happened? Some bad guys got into our
network. They ended up getting a hold of
a service account that had VPN credentials.
And I'm sure some of you are sitting there thinking service
account with VPN credentials. Yes, I'm not sure why that
particular one has the credentials it had.
(06:37):
And I, I had overlooked doing the full account audit when I
first got into the seat because I was trying to just get my head
around everything with the, the network because I did go from
campus technician into the, the network operations manager.
So when I got into that seat, you know, everything opened up
and I'm like, oh, there's a lot of stuff going on here and I
just hadn't gotten around to it at that time.
(06:58):
And the incident itself happened, what, about six months
after I'd gotten promoted into that position.
And it happened in January, right after the new year.
We were one of those stories. So they had gotten in, they
compromised our VM infrastructure.
There was the ransom note on there.
They they were able to get the entire Esxi cluster, but they
(07:22):
weren't able to get the VMS themselves.
And then they were trying to jump around between our other
domain controllers when we noticed everything was actually
like a cyber incident. As you can see, there were five
different servers that got access.
Three of them were domain controllers, two of them were
our backup and replication servers.
(07:42):
And that too were also includingthe Sxi cluster, just because we
did actually restore from that guy some of the immutable
backups from it. We're also still confident that
no sensitive PII or any or any of that type of data was
exfiltrated. I'm sure that they got some
directory information just from our AD structure and everything.
(08:02):
But as far as like account details and everything like
that, passwords have already been changed.
MFA's on everything. But we were monitoring the the
dark web along with our securityteam and we didn't see anything
relating to to our district at all.
And obviously we weren't going to go kind of calling out who
the the people work is. We're not trying to have that
(08:23):
smoke. Next slide is going to be the
timeline of what happened. So I remember this day clearly
because my truck was actually inthe shop.
So I was, you know, having a remote day, right?
It was nice rolled out of bed, starting to check some emails
while I was kind of waking up. And that's when I started seeing
some, I saw a couple crowd strike alerts that were kind of
(08:43):
like, I don't like that it's doing something it shouldn't.
So I was kind of looking into that.
And then we had gotten alerts ofsome network outages and that's
when it was like, OK, cool, I'm keeping this in my mind, but
we're looking into what's going on here.
And we lost access to our our V Sphere controller to actually
see all of our VMS to see what was going on with the VMS.
(09:06):
We ended up trying to reboot oneof the boxes and that kind of
kicked everything off. So we had called up one of our
network consultants at the time,and I was sitting on the phone
with them. We're going back and forth
trying to get back into the box.And then Crowdstrike started
sending some more alerts. And that was when the threat
actor was actually on our domaincontrollers because they lost
access to the boxes they were trying to get into at the time,
(09:29):
which is what we were also trying to get back into.
So they decided to start pivoting around and Crowdstrike
was, you know, setting up all these alerts.
It was blocking stuff on the domain controllers and when I
saw those emails come in, that'swhen I was like, Oh no, we are,
we are toe to toe with bad guys right now.
So I had one of my guys that wason site at the time just kill
(09:51):
off VPN access entirely. We just cut everything to the
everything to the outside world.Essentially.
My boss came and picked me up and by the time I got to the
office around 8:45. So this all started between like
5:30 to 8:45. By the time I got into the
district office, the the bridge that initially started with
(10:13):
myself and our network consultant had about 40
different people on there ranging from people from his
company. We had DHS on the line.
I want to, I want to say the FBIwas on there for the good
portion of the day. Our insurance company was on
there and there's a couple otherrandom, random people here and
there throughout, but there was a lot of people on that call.
(10:36):
And when I got when I got in, that's when I kind of started
dividing up the the roles of who's going to be doing what.
Making sure that my guys are working with, you know, the sys
admin team from the incident response team, making sure that,
you know, the feds are getting the information they need.
We're getting information from them and the support from them
that we need and just kind of bein that like command and control
element for the entire process, which later on I'm going to
(11:00):
explain how that was one of the big reasons aside from the the
no data exfiltration of why it cost us so little for recovery
stuff. We actually were able to get all
critical services backup just before midnight that that same
day because we did, we were ableto restore from our immutable
(11:21):
backups. We did however lose some of our
servers which belonged to Luke and his team, which caused them
some Kartik for I wanna say at least a month and a half back.
After you back up all your servers.
That's my like 1 technique. Like all of them.
Like just don't. If you add a new server, make
sure it's part of the backup. Yep.
Yeah. We have refined our processes
(11:41):
and policies to make sure that all servers that need to be
backed up are getting backed up cuz we have a couple that don't
need to be. But we have a continuation of
the timeline on the next slide. This is this is kind of like
that long that big picture of the timeline of what happened.
So day one that was, you know, obviously the investigation,
(12:02):
we're finding out what happened.We're trying to get services
restored. We were also kind of letting
people know the people that needed to know what was actually
going on knew what was going on.And then we had let all staff
know that we were having some network issues just because we
wanted to make sure that we knewthe full scope of the incident
before we started kind of raising alarms and everything
(12:23):
like that. I don't like making panic for
like unnecessary panic. Day 2, we had the network
restored. So this happened.
I want to say we're going into the weekend.
I can't remember exactly the, the days.
It's, it's been a while, but we had gotten the network fully
restored. School was able to happen the
very next school day. You know, they were still being
(12:47):
told it was, it was just a network outage.
There were still services that were impacted, obviously.
So they were kind of rolling with it.
The the rumor mill had started already just because the
notifications were outside of the normal, because usually I
let them know pretty quickly when stuff is happening.
But that day it was all hands ondeck and they didn't actually
get a notification until the later on in the evening.
(13:10):
So rumors had already started going.
Day three to six, we were kind of auditing logs.
We were rebuilding DNSA lot of the because our DNS servers got
hosed during that incident as well.
They were they were not caught in backups.
That was that was really fun. But we're going through digging
through all the audit logs. We were digging through all the
servers. We were making sure that there
(13:32):
weren't any additional hooks. We were confident that the first
day we got all the hooks just out of there.
There was only the one account we saw that was doing stuff.
We naked that account, which I'll get into that a little bit
or a little bit later on as well.
And we're just kind of just digging through trying to find
all the information we could while also continuing to shore
(13:54):
up our defenses. Day 7 is when we actually let
them know what happened and we have AI believe an example
e-mail on the next slide that will that will show and all
these slides are available in the link.
We dropped the link in the in the chat and then at the end of
the slide they'll be as well day10.
(14:16):
So so with the student passwords, that one was fun.
I won't go into the details about how our stuff is all set
up with that, but essentially wehad to change all 10,000
something student passwords. It was definitely AI.
Will they? Undertake someone asked in the
live thing they said why did youhave to change those student
passwords and the answer was that our student passwords were
(14:39):
set as the description field in our all of their AD accounts in
plain text and that was an existing infrastructure that we
have since changed. Yeah, at the time when it was
set up, it was an archaic account automation script.
It worked. It did what it needed to do.
However, we needed something secure and less, you know,
(15:04):
archaic. It's the same, same old story
that most IT professionals have come across at one point or
another. And then by day 21, we had the
all the servers that Luke was incharge of her.
Most of them I think by that point had been rebuilt and were
functioning in most of its entirety.
(15:25):
So next slide, this is going to be the example e-mail that was
sent out to staff. I'll leave it up for for a
moment. That way y'all can scan through
it. I think an interesting part of
this that is a really important feature of this is not over
sharing things. We didn't want to expose
ourselves to any liability. We didn't we've again, we've
(15:46):
avoided the word cyber attack. We never made any reference to
to data believing or any any sort of breach anything like
yeah, we never used the word breach.
And this is all under the guidance of our insurance and
the, the trust, the the folks that supported us through this
homeland, all that. But it was also really important
(16:10):
to let the community know what was going on.
So striking that fine balance. We had a a lot of communication
back and forth with our leadership and our, you know,
the balancing the IT realities with what the leadership wants
to communicate and needs to communicate for the well-being
of the community. It was lots of back and forth
there. It was done by other members of
the team that you know, aren't apart of this right now, but
(16:30):
we're really clutch with everything publicly.
One thing to note with with the correspondence is I didn't
actually have a big part in how the this the letter or the, the
emails were sent out. But I did have a big part in
the, the verbiage that we used that was used so they would send
the the e-mail to me. So I can just kind of scan
(16:51):
through it real quick because they wanted to make sure they
were using the proper terminology just because as the
subject matter experts, we should be giving them the proper
information to make sure that they're one, giving the right
information and two, not hittingany of those weird kind of like
legal like boundaries that exist.
So working with your legal team,if you have one, definitely do
that. Then you can pre stage a lot of
(17:13):
these different emails. So you have like your draft
templates already kind of like ready to rock'n'roll.
So now we'll kind of get into the the technical details.
Excuse me. So how do we determine no data
exfil or data got exfiltrated? The biggest thing is we didn't
see it up for sale. We were watching the we were
monitoring the dark web, you know, the different little like
(17:34):
channels that the threat actors will use to sell this kind of
data and everything. We didn't see anything.
Our security team that we were working with didn't didn't see
anything. And they still poke around every
now and then and there hasn't been any mention to us or this
attack. We've been kind of keeping up,
keeping tabs on it. Like I said, the directory
information I'm sure did get getexfiltrated, which is why we had
(17:56):
to do that password change for all of our students just because
of the way that that script had worked.
So who is involved in the response and recovery?
My entire district office tech team was there.
So we had our sys admin, our network admin, and our special
special system admin. We were all kind of working
together. I had everyone split up.
(18:18):
Our campus technicians were essentially running screens at
the sites they'd had been given,gotten clued in very briefly and
very high level and told to keepit on on the low.
As far as they were aware of it was network issues that they
were telling people it was so they were just trying to help
kind of alleviate the the campuses while we did our work
(18:38):
up at district office. For external, we had the Arizona
Department of Homeland Security.We had our network vendor who
also had a security team that we're working with and we still
have them on desert kind of a sock as a service, if you will,
just helping us monitor our Crowdstrike environment and make
sure things are still good within our our infrastructure.
(18:59):
The FBI was on the line and thennot mentioned.
We also had our insurance, our insurance company, the trust was
on there as well. And I believe there are a couple
other state entities that that popped in as as well from like
the governor's office, but they didn't interact with much, which
is why I don't really remember the next bullet point insurance.
So we didn't have MFA across theboard.
(19:23):
We had it on a bunch of different things, but we were in
the process of building out thatimplementation.
And one of the problems was the network infrastructure.
While it worked, it wasn't set to be able to be modernized.
So we were in the process of also having to modernize the
network while also, you know, building up the defenses.
So we were in the process of implement, implementing MFA.
(19:45):
And unfortunately, the servers and the account that got hit
weren't trying to do that MFA. And actually, I run, no, I run
that back, sorry. The, the reason why they didn't
cover it was because our firewall couldn't support MFA.
That was what, what didn't have the MFA and then it wasn't on
the servers at the time as well,but the firewall at the time
(20:09):
was, it was a very outdated model.
It was still running as it needed to be, but it it wasn't
talking with. All of its components, because
you had your chassis, the, the brains and then the controller
and they were like kind of talking but not fully talking.
So it wasn't creating logs, it wasn't doing like the latest
(20:30):
like threat lists and everythinglike that.
And that's, that was a big problem that we, I did.
It still makes me kind of kind of frustrated at it, but we
weren't able to see exactly how they got in and like how long
they were in, how many times they got in just because those
logs didn't exist on on the firewall side for that VPN.
(20:54):
But the insurance saw that and they're like, hey, we're not
covering any of this. And we're like, all right,
sweet, this sucks. Let's keep going because we got
a job to do. So we kept running through.
Like I said, I was acting as thecommand and control element.
So I was making sure everything was just kind of running,
rocking and rolling. Our IR team had everything they
needed that we could provide anyways.
(21:14):
Our teams had everything that they needed and the support that
they needed from external and wewere able to minimize the cost
that way. Overall, I want to say it was
under $100,000 from recovery to our sock as a service kind of
deal. And like I said, that was just
(21:35):
because I was taking charge and we didn't get data exfiltrated.
If one or the other didn't happen.
It would be a little bit different if we didn't have the
command and control piece, project managers would had to
been brought in. So that would have been a bunch
more extra money, but that data got exfiltrated.
It would have been a typical, typical ransomware event that
you see. We got lucky on that front.
(21:59):
Our 321 backup method was both part of the problem and the
solution. Part of the problem being it was
the service account for our backup solution at the time that
got compromised now. So you might be sitting back and
saying wait the service account was for your backup solution?
Why does it have VPN act or credentials?
(22:23):
My thoughts exactly. And that's that runs back to
that account auditing. Since then, I've gone through,
we've audited all of our serviceaccounts.
They, they have what they need. And then if they don't need to
be logged in. We also have a, a group that
we've applied to a lot of the different service accounts where
they don't have interactive lockinsurance anymore, just they're
just not able to. And we are going to be moving to
(22:45):
the group managed service accounts at some point.
But we still have infrastructurethat we have to upgrade on the
back end before we can get to that point.
And we're, we're working on it next slide.
All right, so this is this is what my boss did during during
this event. And it was mainly to keep
everyone away from me and keep the ball rolling on the on the
(23:08):
higher level stuff. So we pre planned expectations.
And this was this was really pushed by me.
She just facilitated getting getting me in front of everyone
that needed to to hear this. So we actually had a
presentation and I have an example slide on the next slide
from that presentation where we pretty much showed them the hey,
it's it's if not or it's when not if we get hit.
(23:31):
And they were just kind of sitting there like, what do you
need? Kind of dug into the, you know,
there's a million and one different ways to get through,
get in and kind of ran through like how it would impact.
But then there's a plan with it with that.
When I got in front of them, it wasn't just a scare, scare
method. Hey, give me money.
It was, hey, things are going tohappen.
(23:54):
Here's how it's going to play out.
You know, and there's this big flow chart, the flow charts also
in the presentation. So it's coming up in right after
the, it's right after the next slide, which will show kind of
the high level of what happens during a cyber incident.
And ours panned out exactly likelike the flow chart.
It's just we had a bunch of different things going on at
(24:15):
once because we had different teams all working together.
The efficient flow in keeping the team small, that was that's
going to be for your executive teams.
So she put in there, if you haveyour chief of staff, your
Superintendent, your executives,anyone that has decision making
authority or any of your higher level cabinet members, you're
going to want them on the team because you're going to want
(24:36):
everyone's, you know, involvement and input during
this event. Because they're also going to be
helping keeping their people calm and their people, you know,
understanding like, hey, it is working on it.
We got to do what we can to, youknow, help them, right?
And sometimes that's just by notdoing anything at all.
Having a folder of pre written releases.
(24:57):
This was recommended by our lawyer and then also by me.
That way we can make sure that even if there's like absolute
chaos, everyone, everything's onfire, we don't have to worry
about, you know, potentially saying the wrong thing and then
we have lawsuits coming or we have people saying, hey, well,
what's going on over here? So pre stage a lot of this stuff
(25:18):
and AI can help with a lot of itas well.
At this point, depending on obviously depending on
regulations in your industry or your organization, yeah, start,
start now, start, start pre staging things and getting
things ready. So that way when the chaos
happens, you have all your equipment ready to rock'n'roll.
(25:39):
All right, so the next slide will be the example from our
kind of like, hey, we need to beaware of this.
And I briefed it just as it was written.
And I put the knee jerk reactionin there just because I do know
that when people get panicked when when they're in the hot
seat, knee jerk reactions kind of become a come a thing.
But they don't always help. Remaining cool, calm, collected,
(26:00):
that is #1 you got to, you got to keep your cool.
And if you're not able to, that's perfectly fine.
Find someone who is and put themas the the incident lead.
Because you need that person who's going to be that central
point of information to be cool,calm, and collected when
everything's on fire. The second bullet is something I
(26:20):
say a lot. Trust the process.
You can't always trust the process if you didn't, to make
sure the process is good. But with the process that I'm
referencing, it's going to be our cyber incident response
plan. And I just keep reiterating that
to our executives because they've seen the plan, they've
seen it in action, They know that it works.
(26:42):
So when things start kind of getting ramped up, we just say,
hey, remember, we have things inplace to help with this,
supporting the incident commander.
That's a big thing that we needed from the executive
cabinet and they were, they wereall on board with it when we
briefed it, we went up there, wedecided, hey, this is how things
will pan out. Boom, boom, boom, boom, boom.
(27:02):
And then by the end of it, they did have some great questions
and then they're like, hey, listen, the district's in your
hands. And I was like, all right, cool
number or the 4th bullet followed disaster recovery plan,
you know, just kind of that stuff and that highlighted
bullet right there. That's so that highlighted
bullet right there. The we provide scripts and
templates of what to share. That goes back just to making
(27:24):
sure that the right terminology is being used.
Because even though like you have some brilliant people that
could be working in your executive suites, they might not
know the difference between a breach and an incident, right?
They might not notice or know the difference between like
Adidas or fishing, you know, right.
So you just want to make sure that when you're providing them
(27:45):
the scripts and the templates that it has the proper
terminology and what they're going to need.
So they have minimal stuff that they have to do.
That way they can focus on otherstuff.
The other thing this is, this isa me directive.
Do not pay the ransom. I don't negotiate with
terrorists. If your organization does, then
follow the organization's guidelines.
(28:06):
However, I do not. Well, I followed my
organizations guidelines, but I don't pay ransoms.
This slide right here. This is the flow chart that I
made. It does have my contact
information down at the bottom. The e-mail is going to be the
best way if you have follow up questions.
That phone number, it's mainly used as a voicemail trap for
sales calls or if I need to place calls.
(28:28):
But this, this flow chart is kind of what my brain envisioned
on the paper for other people tobe able to follow.
And I'll give everyone a moment just kind of look over it.
It did end up panning out prettymuch exactly as I I listed it
with little adjustments here andthere.
(28:49):
And then this, there is a a PDF copy of this in the slide
resources and there's also an editable copy.
I did make it in Visio. So it's a VBDSX file, I think.
I think that's what the that file is.
That way you can just kind of take this and run with it as a
template. Obviously I'm not going to be
your incident commander, so you'll want to change that
information if you do recycle it.
(29:12):
Some of the things that we are are proud of, like the reason
that we treat this as a success,the reason we're willing to give
a presentation about this is because so much went well.
You know, someone found a compromise account and got in.
Yes, that is a bad thing. But everything around that is
pretty strong. Brandon had already prepped our
leadership about it. He had a CIRP, he had the team
(29:34):
in place. He didn't get flustered.
He he followed the process. There were backups, enough
backups in place even though onegot compromised.
There were some things that werenot 100% perfect about it.
He had multiple backups and so we were able to restore really
quickly. All essential services were
backup by the very next day, which is amazing.
(29:55):
Great. I think I just said all of those
low recovery costs. OK.
So one really key part of this is that Brandon again has the
knowledge and expertise because he has been in the industry long
enough that he we didn't have tooutsource a lot of the things he
keeps saying that, you know, he was able to act as the the
response command of this. That keeps costs really low.
And so when we debriefed with our vendors and the support that
(30:17):
we had afterwards, that was something that they definitely
noted. They're like, they didn't have
to provide someone else to be that link between everything
because he was able to be that. So sorry, everyone, we aren't
sharing, but Brandon was a really valuable part of that.
I mean, that would be part of our conclusion is like, what do
you do if you don't have a Brandon?
Brandon, do you feel stable enough?
Do you Anything else you want toadd that I didn't cover from
(30:40):
those? No, not for you.
You hit it all on the head. All right.
So for sure, though, yeah, yeah,yeah, I got this one.
I think I thought audios, the APstuff figured out.
All right, So what do we improveafter obviously our backup in an
account auditing, There was a tool I use, and I have to say it
slowly because it definitely sounded really funny when I said
it fast. It's called Ping Castle.
(31:03):
I'll put that in the chat as well.
That's a tool I used. It's a open source tool just to
kind of audit my Active Directory environment.
Puts a whole bunch of different information, but I was able to
see which service accounts I had, which ones had permissions
to wear it was it was great. We also have a different backup
(31:24):
solution. It's not that our the backup
solution time was a bad one. The product itself is good, but
it required too much work on ourend for the size of our team.
So we went with something that'sa little bit more like hybrid
cloud managed. We also went through and cut
down a lot of our service accounts and scaled the back,
made sure that they, if they didn't need to be logged into,
(31:48):
they weren't able to be logged into.
None of them have VPN account orcredentials.
All of them are locked behind MFA.
All of our servers are behind MFA, you know, staff accounts,
all that good stuff. So we, we went through and
tidied up all of our account stuff.
We refined our cyber incident response plan because you always
(32:08):
have to keep refining it. It's not something that you just
make once and you just let it sit for 27 years.
It's almost like every other month you got to be looking at
being like, oh, now we got to kind of modify this.
We got to change this because technology is changing.
So make that a living document. We went through and upgraded our
firewall. We got a brand new firewall,
completely different vendor. Again, not because the product
(32:29):
we had was a bad one, but super outdated.
We wanted something a little bitmore modern and could kind of
work with the setup that we weregoing with because we were at
the time and still are making some back end changes on our
infrastructure itself to kind ofmake things more redundant and
just flow a lot better and be able to handle everything
realistically because we want togive the students the best
(32:49):
experience possible. So we added in a security vendor
to augment our team sock as a service.
They've been great with us. They went through in, they added
in a bunch of different rules and stuff into our crowd strike
environment. They help make sure that you
know, it's getting what it needsto and then they'll poke in when
alerts are kind of coming through and verify if it's a
(33:10):
false positive or not. It's, it's been great.
And then we'll, we'll see stuff and ask them questions and we'll
kind of like jump in, go back and forth.
Already mentioned the removing interactive logins.
We did that through group policy, by the way, on the
Windows side for our service accounts.
And again, if, if you have questions on any of the stuff I
mentioned, my e-mail is available.
(33:33):
We also went through and patchedall the stuff.
We have Tanium for patch management, which I don't think
I mentioned this, yes, but yet. But the Arizona Department of
Homeland Security has a program called the Cyber Readiness
Program where they provide a bunch of security tools at no
cost to state and local education entities.
(33:55):
So we we have some products through them.
So we're getting our Crowdstrike, Titanium and Duo
MFA through a CDHS. It's a great program.
I love it. Otherwise we, we would be not as
far along in our network stuff and security kind of bolstering
as we are now. Next slide.
Ah, yes, so, so when we before we had actually presented this
(34:20):
time at or this presentation, wehad another attempted incident
where they had gotten in throughthis was through another
account. And we're still in the process
of adding VPN into the M or intothe firewall.
Cause the firewall was just put in place and we're still kind of
building everything back out. So they came in through the VPN,
(34:43):
but they were in for less than 15 minutes.
Crowd strike started popping some alerts and blocking stuff.
And then we actually saw that VPN connection in the, the new
firewall and we saw that it justtimed out and there was no
reconnections because we were, we were monitoring for
reconnections on that account. But they, they gave up.
(35:04):
They saw crash strike was on. There was like, Nope, we're not
doing this. So we did go through and, you
know, change that password and everything like that to make
sure that it got under the proper MFA.
But yeah, that one, that one wasjust to show that, hey, what we
did helped. And we definitely did bring this
up to cabinet. We said, hey, look from that
(35:25):
incident that happened, here's you know what we've implemented
and changed, here's what just recently happened and here's how
far they didn't get, which was amazing all.
Right. And this is my cue to take back
over. So you know, in the in this live
session that we gave some chanceto like discuss in groups, talk
to people around you, etcetera. But I hope that this, like our
(35:48):
walkthrough of both what happened and what we did before
and after that made it valuable and something that we get to
brag about has given some information, some ideas that you
could be able to use with your teams.
So these are some questions thatthat should be useful for you to
think about if as you as you consider preparing or auditing
(36:08):
or responding your own cyber incident response plan.
So what would you want to do? These are a lot of the things
that are the practices that we either have or learned or
improved. Make sure that you always know
which accounts exist, what they can access, trim off anything
that is not needed. It is not the most fun task, but
(36:30):
it is one of the most valuable ones.
You know, showing up all of the little the cracks in the wall
that keep the bad guys from getting in.
We really are are grateful that we did that leadership
presentation to let them know that it's not if, but when so
that they weren't startled when it happened.
You can use us as an example of folks who like we, we tried and
(36:51):
we prepared. It still happened to us.
There's nothing you can do the the bad guys are going to find a
way of some sort to try to do something.
So justify at least like a a good 15 minute presentation on
the state of cyber across the country.
Education is a major place that is targeted by these threat
actors so that they are aware. So it's not the first time
(37:14):
they've heard about it when something occurs to you and then
yes, have a plan. Steal ours.
Steal Brandon's as a start for where you can go from there.
And if you don't have a Brandon I again, we will not share him.
He's stuck here. But there is there are ways that
you can address this without having to hire like if you can't
(37:35):
find someone with this expertisewho wants to work in education,
there are hopefully partnershipswith your state and local
governments. Federal government has things
that are available to you to be able to protects the data of our
students. It's a thing that matters.
Yeah. I mean, getting an MFA in place
everywhere you possibly can is one of the most valuable things.
(37:57):
Brandon, do you have anything tothrow of like funding wise?
Like what can you do if we have limited funds and we don't have
a brand and like what would yourwhat do you got?
I mean, the, the difficult part is not having just that kind of
like just the, the cyber knowledge of just all the
(38:17):
different possibilities of what is and, and isn't, but have a
plan. You can find a whole bunch of
different resources online. And I, I should probably just
make like a little quick, Quick guide of like, Hey, what should
you be checking to make sure your stuff is good?
But the biggest things, make sure your backups are good and
that there are backing up and that you have an immutable
(38:38):
backup. I know there's a, there's new
terminology going around that's better than mutable.
I haven't committed to memory yet, but there are better ways
to do it. But make sure your backups are
good. Make sure your service accounts
are locked up and that your accounts have MFA.
Your servers have MFA, right? And if it, if it is something
where it's where it's like a, a cost thing, make sure your
critical servers have MFA and your critical service accounts
(39:01):
or any accounts that have admin privileges, right?
Because you can look at trying to do, you know, scaled stuff.
It would be better if to have everything with MFA.
But if you're on a budget constraint, you do need to still
try to make some steps forward to secure your stuff and then
have a plan. I mean, the other big thing is
just having a plan and knowing what to do when everything kicks
(39:21):
off. You can use that flow chart if
you want. You don't have to reinvent the
wheel because there's 27,000 people random number that have
made the same presentation that we've given, I'm sure.
But there's, there's a lot of resources.
You don't have to reinvent the wheel, but you can take stuff
and tailor it to your organization.
(39:41):
But you definitely want to have a plan.
You always need to have a plan at the bare minimum.
And thanks for sharing that linkin the in the chat to Kosan
Kosan's website about cybersecurity resources because
free resources, a great robust network of folks who can
contribute some to that. So that is another next step
(40:03):
after this. OK, we got to get our house in
order. Let's start working through some
of this last slide. We'll you know.
We will not walk through this, but here's six different things
to do, some sort of audit, like,all right, over the next six
months, what is one or two stepswe can do to make every single
one of these just one notch better than they were at this
(40:23):
moment, right? This can be some sort of
catalyst, some sort of call to action for you and your team to
say, all right, we don't have todo everything.
We can't hire a Brandon, but let's make this.
Let's have a CIRP, let's add, let's audit our backups and make
sure it touches everything or make sure, you know, make a
third backup. So if you know you're hot, warm,
cold, like do we have all of these?
(40:44):
So hopefully that can be something that you can take some
good action steps on. If you do have some questions,
you can throw them in the chat. It's been available the whole
time. Here is our final contact us
slide. That QR code and the Bitly there
will get you access to the resources that we've provided
(41:05):
the the sample communication that we sent out to the
community, the incident responseplan, etcetera.
And I wouldn't have a lot to addif you emailed me, but I bet
Brandon would. So yeah.
Yeah, feel free to e-mail me with any other additional
questions. And there was, so I tried to
remember at the, at the end of our panel at the Coast in
(41:27):
25/20/25 in Seattle, there was abunch of great questions.
And unfortunately, I can't remember all of the, the
questions that we had. It was great back and forth, but
there was one that seemed to be a recurring topic and it was how
to get buy in from, you know, your executives and everything.
And that one is pretty difficult, but there's some
(41:48):
things that you can do to kind of help.
You don't have to be a people person either.
You just need to be able to speak in their language.
So one of the OR a couple of things you can do to help get
yourself by and as you're kind of developing this plan, right,
you want to put in, in the business terms.
So you want to put in terms of money, dollars and impact,
right? How is this going to impact
(42:09):
either funding budgets, wallets,you know, the money stuff,
right? Or how's it going to impact the
service that you provide? And if you're working in
education, the service you provide is education to the
students, right? So how is that going to impact
them, right? Well, if the network's down for
an entire day, that means that'san entire day that the students
don't have access to the learning resources, right?
(42:31):
So you have to, you have to put it in terms that they're going
to understand. That's, that's the, that's step
#1 #2 have a plan ready before you get the buy in right.
So if you're going to say, hey, listen, there's these bad guys
that are going to, they're always trying to get into our
network. Here's some stuff that happens
from our logs and then you just leave with that.
You're probably not going to getthe money you need.
(42:52):
But if you're like, hey, listen,here's some trends that we see.
Here's how it impacts. Here's how we're here's some
things that we're doing now already, right, to help kind of
prevent. Here's some things we want to
do. Here's some, you know, do like a
short term, like, you know, like3 to 5, you're kind of like
window, right view and say theseare things we want to do.
However, we need funding to get some of these.
(43:13):
And here's how each different thing is going to, you know,
positively impact or negatively impact if you don't get the fuck
right. Put it in terms that they can
understand so they can visualizewhat actually happens.
Because a big time or a big problem is with that translation
from technical to non-technical is non-technical.
Can't necessarily like envision all the technical stuff like we
(43:35):
can just because, you know, we've been working in the fields
for so long. We kind of just we know things
they don't know those things that we do.
So we need to put that in their terms, show cost analysis.
If you're able to get a cost analysis going, if you even run
over to your business departmentand say, hey, listen, can you
help me make a cost analysis forthis?
I'm trying to like, you know, get some funding, right?
(43:57):
I am sure that that someone in that business office will be
willing to help you create something like that, especially
if you show interest in that, like their job about it because
people want to talk about their job, right?
So kind of just build that relationship.
You'll be able to start getting a little bit more buy in from
just, you know, people in the inthe departments as well.
(44:18):
That does go a long way. Another thing, transparency,
keep your staff informed. You don't have to tell them all
the nitty gritty. You don't have to tell them all
The Dirty details, but give them, just clue them into what's
going on. If there's network issues going
on, just be like, hey, we're experiencing some network
issues, blah, blah, blah. Over time that does have a very
(44:38):
positive effect because I've been, I started doing that when
I was a site technician at the campus I was at and I still do
it now. It's, it's a little bit
different how I do it now. It's not just sending out all,
all staff emails on blast. We're, we're going into a more
newsletter kind of format just so that way we reduce the amount
of scrolling the staff has to do.
We can keep it like really shortand condensed, but we just keep
(44:59):
them in the loop on what's goingon and when staff has left our
organization. I, I haven't personally heard
it, but my colleagues have wherethey'll be chatting with former
staff members and like, yeah, it's like night and day with the
IT department in terms of just knowing what's going on because
they would go from us to anotherdepartment that doesn't keep
(45:21):
their people in the loop. So just that little bit of
transparency goes a very long way in building that trust
between your staff and your higher ups.
Yes, that's that's about all I have for the the buy in off the
top of my head. Yeah, if there's any other
questions definitely drop them in chat.
If not, y'all have a wonderful, wonderful day there.
(45:43):
You go. Thanks everyone for
participating. Get ready to connect, learn and
lead at Kosin 2026, happening atthis April in Chicago for the
future of education through the lenses of AI, cyber security,
(46:04):
emerging technologies and more. You'll hear from visionary
speakers, dive into hands on sessions, and discover solutions
that move learning forward. All all while networking with
the brightest minds shaping K12 at Tech today.
Don't miss your chance to be a part of the conversation driving
human leadership in a world of AI.
Register now and say with early word rates, visit cosin.org.
(46:28):
That's cosn.org.
All right, let's kick it off everyone.
My name is Luke Alpress and and this is Brandon.
We will introduce ourselves morein a little bit, but thanks for
attending or reviewing this Coast in webinar series on
follow up from Coast in 2025. So this is a session that we
gave back in the Seattle Coast in conference and we're excited
to share it again with you in this webinar focus with webinar
(00:28):
medium. Hopefully they can just give an
idea of what it went it was likefor us to go through.
Yeah, cyber incident, introduce ourselves really briefly.
One of us is a former teacher and acedo and like overseas,
like data privacy and presentation.
One of us is an Army vet who fights off the bad guys.
And I don't know if you can see the two of us already, but you
(00:49):
might be able to guess which is which.
I taught math and physics and all those things for a while and
now I, I, I work in the IT department, but adjacent to it,
just helping data provisioning and data engineering analysis,
things like that. And over to Brandon.
Yeah, I'm the the Army vet. Really, I just wanted to blow
stuff up when I was a high schooler, but then I ended up
(01:11):
doing computer stuff in the Armyand loved it kind of.
And here I am. I just recently finished the
masters in IT management. I've been working in this
district for almost five years now, started in 2021 as a site
technician and just kind of keptworking my way up and that
little line at the bottom right there thrice and chaos.
(01:32):
I was in my prime when when we're going through this,
although it sucked, I I really wish we didn't have to go
through it, but yeah, yeah. I'm glad Brandon's on our side.
This is like an overview of QR code and a bit late to access
any of the resources we reference.
There's like emails that we sentout, like this is what we sent
to the community. This is a reference to our CRP.
(01:54):
All that can be found at those links.
Those are e-mail addresses. This slide that can be found
there as well. And also after the Cosin events
in April, there was a feature done on Brandon in this
incident. And so there's a link to that
article there. Brandon, I didn't tell you I put
that on that slide, but it's toogood of a picture of Marcio.
OK, So when we were doing this live, we threw a mentometer up,
but we will show this is the results we got from the
(02:17):
mentometer because we wanted to see who was in the room like
what. Let's calibrate this
presentation to experience that we have coming in.
And so we wanted to look like maybe the difference between the
individuals in the room that, you know, the CTOS, the cyber
experts who are really excited to hear about a cyber incident
and maybe our perceptions of thereadiness of our educational
(02:40):
institutions as a whole. So personal cyber knowledge
people maybe humbly didn't like fully rate themselves all the
way up at 5 stars, but you can see a pretty good nice hump
towards the higher end of things.
Like, Yep, I, I think I know what I'm talking about.
I think our personal infrastructure is good.
Then we get over to like, all right, how's the rest of our
leadership? Do we trust, Do we trust our
(03:00):
staff to not click a phishing link?
You know, and that one definitely got further towards
the less prepared side. And then the very the bottom one
was a fascinating like a very flat like do we have a CIRP full
like that was a across the boardanyone my land anywhere on that.
And so that was a big part of our story.
So Brandon will definitely get into that later.
(03:22):
Less fun to do this in a webinarformat, but we don't we are all
we all share the same woes of like turning to something off
and on again is the right solution.
Has someone sent you the password in the body of an
e-mail? Right.
We have all lived in this IT world and the Internet is broken
only to find out something is unplugged.
(03:44):
OK quick introduction just to set the stage.
I would imagine a lot of the folks that are interested in
attending this webinar will knowsome of this intro content.
But just to at least do some level setting and get us to the
place of having some common language and just getting us all
in the mindset. Education is the second most
impacted industry from ransomware attacks.
Ransomware being the bad guys come in, they try to lock you
(04:06):
out and say, pay us with Bitcoinor something comparable to get
your data back. And government education and
healthcare are targeted because we have lots of data that we
care a lot about and we don't necessarily have enough money
just flowing around everywhere to spend on a really strong
cybersecurity infrastructure. So it makes us susceptible to
that. Today we're going to talk about,
(04:28):
I mean obviously the the title of the session is we there was a
cyber instant aid attempted cyber attack and I always want
to be really careful with this language.
We'll talk about what went down there, what we did to response,
what we learned. And a big part of it is how we
were prepared for it because Brandon did a lot of things that
made us come to this successfully because of the pre
(04:49):
work he had done there. I will give a brief layman's
analogy of what happens in an attempted ransomware attack from
our perspective. So if our data warehouse, if our
on site data can be viewed as a house, the bad guys find some
way to get in. They find a spare key that was
(05:10):
left under the mat. You know, your, your kid tells
the the neighbor down the streetwhere they can find a spare key
and what the code to the security system is just because
they thought it was their friend, You know, so the bad
guys get in and then they can start going maybe into different
rooms of the house. And hopefully we have a really
good security system that says, hey, someone has entered the
(05:30):
house with suspicious behavior of some sort.
Hey, someone has entered this room of the house being a
different server with suspiciousbehavior.
And then ideally the security walls come down.
They prevent them from leaving, but they might still be in the
house. So then we still got to monitor
what they're doing, but then we get do get to track and see if
anything has left the house because we have monitoring on
all of the doors and windows to see what goes in and out.
(05:53):
But the goal being that we prevent them from taking
anything out of the house that would be exfiltration.
And then also in a ransomware attack specifically they're
gonna go in there and try to change the locks on all the
doors and say we will give you the new key if you pay us for
it. So just with that kind of as our
layman's analogy metaphor, I will toss it over to Brandon who
(06:13):
will give more of the technical details that I would imagine
most people watching us are looking for.
All right, So what happened? Some bad guys got into our
network. They ended up getting a hold of
a service account that had VPN credentials.
And I'm sure some of you are sitting there thinking service
account with VPN credentials. Yes, I'm not sure why that
particular one has the credentials it had.
(06:37):
And I, I had overlooked doing the full account audit when I
first got into the seat because I was trying to just get my head
around everything with the, the network because I did go from
campus technician into the, the network operations manager.
So when I got into that seat, you know, everything opened up
and I'm like, oh, there's a lot of stuff going on here and I
just hadn't gotten around to it at that time.
(06:58):
And the incident itself happened, what, about six months
after I'd gotten promoted into that position.
And it happened in January, right after the new year.
We were one of those stories. So they had gotten in, they
compromised our VM infrastructure.
There was the ransom note on there.
They they were able to get the entire Esxi cluster, but they
(07:22):
weren't able to get the VMS themselves.
And then they were trying to jump around between our other
domain controllers when we noticed everything was actually
like a cyber incident. As you can see, there were five
different servers that got access.
Three of them were domain controllers, two of them were
our backup and replication servers.
(07:42):
And that too were also includingthe Sxi cluster, just because we
did actually restore from that guy some of the immutable
backups from it. We're also still confident that
no sensitive PII or any or any of that type of data was
exfiltrated. I'm sure that they got some
directory information just from our AD structure and everything.
(08:02):
But as far as like account details and everything like
that, passwords have already been changed.
MFA's on everything. But we were monitoring the the
dark web along with our securityteam and we didn't see anything
relating to to our district at all.
And obviously we weren't going to go kind of calling out who
the the people work is. We're not trying to have that
(08:23):
smoke. Next slide is going to be the
timeline of what happened. So I remember this day clearly
because my truck was actually inthe shop.
So I was, you know, having a remote day, right?
It was nice rolled out of bed, starting to check some emails
while I was kind of waking up. And that's when I started seeing
some, I saw a couple crowd strike alerts that were kind of
(08:43):
like, I don't like that it's doing something it shouldn't.
So I was kind of looking into that.
And then we had gotten alerts ofsome network outages and that's
when it was like, OK, cool, I'm keeping this in my mind, but
we're looking into what's going on here.
And we lost access to our our V Sphere controller to actually
see all of our VMS to see what was going on with the VMS.
(09:06):
We ended up trying to reboot oneof the boxes and that kind of
kicked everything off. So we had called up one of our
network consultants at the time,and I was sitting on the phone
with them. We're going back and forth
trying to get back into the box.And then Crowdstrike started
sending some more alerts. And that was when the threat
actor was actually on our domaincontrollers because they lost
access to the boxes they were trying to get into at the time,
(09:29):
which is what we were also trying to get back into.
So they decided to start pivoting around and Crowdstrike
was, you know, setting up all these alerts.
It was blocking stuff on the domain controllers and when I
saw those emails come in, that'swhen I was like, Oh no, we are,
we are toe to toe with bad guys right now.
So I had one of my guys that wason site at the time just kill
(09:51):
off VPN access entirely. We just cut everything to the
everything to the outside world.Essentially.
My boss came and picked me up and by the time I got to the
office around 8:45. So this all started between like
5:30 to 8:45. By the time I got into the
district office, the the bridge that initially started with
(10:13):
myself and our network consultant had about 40
different people on there ranging from people from his
company. We had DHS on the line.
I want to, I want to say the FBIwas on there for the good
portion of the day. Our insurance company was on
there and there's a couple otherrandom, random people here and
there throughout, but there was a lot of people on that call.
(10:36):
And when I got when I got in, that's when I kind of started
dividing up the the roles of who's going to be doing what.
Making sure that my guys are working with, you know, the sys
admin team from the incident response team, making sure that,
you know, the feds are getting the information they need.
We're getting information from them and the support from them
that we need and just kind of bein that like command and control
element for the entire process, which later on I'm going to
(11:00):
explain how that was one of the big reasons aside from the the
no data exfiltration of why it cost us so little for recovery
stuff. We actually were able to get all
critical services backup just before midnight that that same
day because we did, we were ableto restore from our immutable
(11:21):
backups. We did however lose some of our
servers which belonged to Luke and his team, which caused them
some Kartik for I wanna say at least a month and a half back.
After you back up all your servers.
That's my like 1 technique. Like all of them.
Like just don't. If you add a new server, make
sure it's part of the backup. Yep.
Yeah. We have refined our processes
(11:41):
and policies to make sure that all servers that need to be
backed up are getting backed up cuz we have a couple that don't
need to be. But we have a continuation of
the timeline on the next slide. This is this is kind of like
that long that big picture of the timeline of what happened.
So day one that was, you know, obviously the investigation,
(12:02):
we're finding out what happened.We're trying to get services
restored. We were also kind of letting
people know the people that needed to know what was actually
going on knew what was going on.And then we had let all staff
know that we were having some network issues just because we
wanted to make sure that we knewthe full scope of the incident
before we started kind of raising alarms and everything
(12:23):
like that. I don't like making panic for
like unnecessary panic. Day 2, we had the network
restored. So this happened.
I want to say we're going into the weekend.
I can't remember exactly the, the days.
It's, it's been a while, but we had gotten the network fully
restored. School was able to happen the
very next school day. You know, they were still being
(12:47):
told it was, it was just a network outage.
There were still services that were impacted, obviously.
So they were kind of rolling with it.
The the rumor mill had started already just because the
notifications were outside of the normal, because usually I
let them know pretty quickly when stuff is happening.
But that day it was all hands ondeck and they didn't actually
get a notification until the later on in the evening.
(13:10):
So rumors had already started going.
Day three to six, we were kind of auditing logs.
We were rebuilding DNSA lot of the because our DNS servers got
hosed during that incident as well.
They were they were not caught in backups.
That was that was really fun. But we're going through digging
through all the audit logs. We were digging through all the
servers. We were making sure that there
(13:32):
weren't any additional hooks. We were confident that the first
day we got all the hooks just out of there.
There was only the one account we saw that was doing stuff.
We naked that account, which I'll get into that a little bit
or a little bit later on as well.
And we're just kind of just digging through trying to find
all the information we could while also continuing to shore
(13:54):
up our defenses. Day 7 is when we actually let
them know what happened and we have AI believe an example
e-mail on the next slide that will that will show and all
these slides are available in the link.
We dropped the link in the in the chat and then at the end of
the slide they'll be as well day10.
(14:16):
So so with the student passwords, that one was fun.
I won't go into the details about how our stuff is all set
up with that, but essentially wehad to change all 10,000
something student passwords. It was definitely AI.
Will they? Undertake someone asked in the
live thing they said why did youhave to change those student
passwords and the answer was that our student passwords were
(14:39):
set as the description field in our all of their AD accounts in
plain text and that was an existing infrastructure that we
have since changed. Yeah, at the time when it was
set up, it was an archaic account automation script.
It worked. It did what it needed to do.
However, we needed something secure and less, you know,
(15:04):
archaic. It's the same, same old story
that most IT professionals have come across at one point or
another. And then by day 21, we had the
all the servers that Luke was incharge of her.
Most of them I think by that point had been rebuilt and were
functioning in most of its entirety.
(15:25):
So next slide, this is going to be the example e-mail that was
sent out to staff. I'll leave it up for for a
moment. That way y'all can scan through
it. I think an interesting part of
this that is a really important feature of this is not over
sharing things. We didn't want to expose
ourselves to any liability. We didn't we've again, we've
(15:46):
avoided the word cyber attack. We never made any reference to
to data believing or any any sort of breach anything like
yeah, we never used the word breach.
And this is all under the guidance of our insurance and
the, the trust, the the folks that supported us through this
homeland, all that. But it was also really important
(16:10):
to let the community know what was going on.
So striking that fine balance. We had a a lot of communication
back and forth with our leadership and our, you know,
the balancing the IT realities with what the leadership wants
to communicate and needs to communicate for the well-being
of the community. It was lots of back and forth
there. It was done by other members of
the team that you know, aren't apart of this right now, but
(16:30):
we're really clutch with everything publicly.
One thing to note with with the correspondence is I didn't
actually have a big part in how the this the letter or the, the
emails were sent out. But I did have a big part in
the, the verbiage that we used that was used so they would send
the the e-mail to me. So I can just kind of scan
(16:51):
through it real quick because they wanted to make sure they
were using the proper terminology just because as the
subject matter experts, we should be giving them the proper
information to make sure that they're one, giving the right
information and two, not hittingany of those weird kind of like
legal like boundaries that exist.
So working with your legal team,if you have one, definitely do
that. Then you can pre stage a lot of
(17:13):
these different emails. So you have like your draft
templates already kind of like ready to rock'n'roll.
So now we'll kind of get into the the technical details.
Excuse me. So how do we determine no data
exfil or data got exfiltrated? The biggest thing is we didn't
see it up for sale. We were watching the we were
monitoring the dark web, you know, the different little like
(17:34):
channels that the threat actors will use to sell this kind of
data and everything. We didn't see anything.
Our security team that we were working with didn't didn't see
anything. And they still poke around every
now and then and there hasn't been any mention to us or this
attack. We've been kind of keeping up,
keeping tabs on it. Like I said, the directory
information I'm sure did get getexfiltrated, which is why we had
(17:56):
to do that password change for all of our students just because
of the way that that script had worked.
So who is involved in the response and recovery?
My entire district office tech team was there.
So we had our sys admin, our network admin, and our special
special system admin. We were all kind of working
together. I had everyone split up.
(18:18):
Our campus technicians were essentially running screens at
the sites they'd had been given,gotten clued in very briefly and
very high level and told to keepit on on the low.
As far as they were aware of it was network issues that they
were telling people it was so they were just trying to help
kind of alleviate the the campuses while we did our work
(18:38):
up at district office. For external, we had the Arizona
Department of Homeland Security.We had our network vendor who
also had a security team that we're working with and we still
have them on desert kind of a sock as a service, if you will,
just helping us monitor our Crowdstrike environment and make
sure things are still good within our our infrastructure.
(18:59):
The FBI was on the line and thennot mentioned.
We also had our insurance, our insurance company, the trust was
on there as well. And I believe there are a couple
other state entities that that popped in as as well from like
the governor's office, but they didn't interact with much, which
is why I don't really remember the next bullet point insurance.
So we didn't have MFA across theboard.
(19:23):
We had it on a bunch of different things, but we were in
the process of building out thatimplementation.
And one of the problems was the network infrastructure.
While it worked, it wasn't set to be able to be modernized.
So we were in the process of also having to modernize the
network while also, you know, building up the defenses.
So we were in the process of implement, implementing MFA.
(19:45):
And unfortunately, the servers and the account that got hit
weren't trying to do that MFA. And actually, I run, no, I run
that back, sorry. The, the reason why they didn't
cover it was because our firewall couldn't support MFA.
That was what, what didn't have the MFA and then it wasn't on
the servers at the time as well,but the firewall at the time
(20:09):
was, it was a very outdated model.
It was still running as it needed to be, but it it wasn't
talking with. All of its components, because
you had your chassis, the, the brains and then the controller
and they were like kind of talking but not fully talking.
So it wasn't creating logs, it wasn't doing like the latest
(20:30):
like threat lists and everythinglike that.
And that's, that was a big problem that we, I did.
It still makes me kind of kind of frustrated at it, but we
weren't able to see exactly how they got in and like how long
they were in, how many times they got in just because those
logs didn't exist on on the firewall side for that VPN.
(20:54):
But the insurance saw that and they're like, hey, we're not
covering any of this. And we're like, all right,
sweet, this sucks. Let's keep going because we got
a job to do. So we kept running through.
Like I said, I was acting as thecommand and control element.
So I was making sure everything was just kind of running,
rocking and rolling. Our IR team had everything they
needed that we could provide anyways.
(21:14):
Our teams had everything that they needed and the support that
they needed from external and wewere able to minimize the cost
that way. Overall, I want to say it was
under $100,000 from recovery to our sock as a service kind of
deal. And like I said, that was just
(21:35):
because I was taking charge and we didn't get data exfiltrated.
If one or the other didn't happen.
It would be a little bit different if we didn't have the
command and control piece, project managers would had to
been brought in. So that would have been a bunch
more extra money, but that data got exfiltrated.
It would have been a typical, typical ransomware event that
you see. We got lucky on that front.
(21:59):
Our 321 backup method was both part of the problem and the
solution. Part of the problem being it was
the service account for our backup solution at the time that
got compromised now. So you might be sitting back and
saying wait the service account was for your backup solution?
Why does it have VPN act or credentials?
(22:23):
My thoughts exactly. And that's that runs back to
that account auditing. Since then, I've gone through,
we've audited all of our serviceaccounts.
They, they have what they need. And then if they don't need to
be logged in. We also have a, a group that
we've applied to a lot of the different service accounts where
they don't have interactive lockinsurance anymore, just they're
just not able to. And we are going to be moving to
(22:45):
the group managed service accounts at some point.
But we still have infrastructurethat we have to upgrade on the
back end before we can get to that point.
And we're, we're working on it next slide.
All right, so this is this is what my boss did during during
this event. And it was mainly to keep
everyone away from me and keep the ball rolling on the on the
(23:08):
higher level stuff. So we pre planned expectations.
And this was this was really pushed by me.
She just facilitated getting getting me in front of everyone
that needed to to hear this. So we actually had a
presentation and I have an example slide on the next slide
from that presentation where we pretty much showed them the hey,
it's it's if not or it's when not if we get hit.
(23:31):
And they were just kind of sitting there like, what do you
need? Kind of dug into the, you know,
there's a million and one different ways to get through,
get in and kind of ran through like how it would impact.
But then there's a plan with it with that.
When I got in front of them, it wasn't just a scare, scare
method. Hey, give me money.
It was, hey, things are going tohappen.
(23:54):
Here's how it's going to play out.
You know, and there's this big flow chart, the flow charts also
in the presentation. So it's coming up in right after
the, it's right after the next slide, which will show kind of
the high level of what happens during a cyber incident.
And ours panned out exactly likelike the flow chart.
It's just we had a bunch of different things going on at
(24:15):
once because we had different teams all working together.
The efficient flow in keeping the team small, that was that's
going to be for your executive teams.
So she put in there, if you haveyour chief of staff, your
Superintendent, your executives,anyone that has decision making
authority or any of your higher level cabinet members, you're
going to want them on the team because you're going to want
(24:36):
everyone's, you know, involvement and input during
this event. Because they're also going to be
helping keeping their people calm and their people, you know,
understanding like, hey, it is working on it.
We got to do what we can to, youknow, help them, right?
And sometimes that's just by notdoing anything at all.
Having a folder of pre written releases.
(24:57):
This was recommended by our lawyer and then also by me.
That way we can make sure that even if there's like absolute
chaos, everyone, everything's onfire, we don't have to worry
about, you know, potentially saying the wrong thing and then
we have lawsuits coming or we have people saying, hey, well,
what's going on over here? So pre stage a lot of this stuff
(25:18):
and AI can help with a lot of itas well.
At this point, depending on obviously depending on
regulations in your industry or your organization, yeah, start,
start now, start, start pre staging things and getting
things ready. So that way when the chaos
happens, you have all your equipment ready to rock'n'roll.
(25:39):
All right, so the next slide will be the example from our
kind of like, hey, we need to beaware of this.
And I briefed it just as it was written.
And I put the knee jerk reactionin there just because I do know
that when people get panicked when when they're in the hot
seat, knee jerk reactions kind of become a come a thing.
But they don't always help. Remaining cool, calm, collected,
(26:00):
that is #1 you got to, you got to keep your cool.
And if you're not able to, that's perfectly fine.
Find someone who is and put themas the the incident lead.
Because you need that person who's going to be that central
point of information to be cool,calm, and collected when
everything's on fire. The second bullet is something I
(26:20):
say a lot. Trust the process.
You can't always trust the process if you didn't, to make
sure the process is good. But with the process that I'm
referencing, it's going to be our cyber incident response
plan. And I just keep reiterating that
to our executives because they've seen the plan, they've
seen it in action, They know that it works.
(26:42):
So when things start kind of getting ramped up, we just say,
hey, remember, we have things inplace to help with this,
supporting the incident commander.
That's a big thing that we needed from the executive
cabinet and they were, they wereall on board with it when we
briefed it, we went up there, wedecided, hey, this is how things
will pan out. Boom, boom, boom, boom, boom.
(27:02):
And then by the end of it, they did have some great questions
and then they're like, hey, listen, the district's in your
hands. And I was like, all right, cool
number or the 4th bullet followed disaster recovery plan,
you know, just kind of that stuff and that highlighted
bullet right there. That's so that highlighted
bullet right there. The we provide scripts and
templates of what to share. That goes back just to making
(27:24):
sure that the right terminology is being used.
Because even though like you have some brilliant people that
could be working in your executive suites, they might not
know the difference between a breach and an incident, right?
They might not notice or know the difference between like
Adidas or fishing, you know, right.
So you just want to make sure that when you're providing them
(27:45):
the scripts and the templates that it has the proper
terminology and what they're going to need.
So they have minimal stuff that they have to do.
That way they can focus on otherstuff.
The other thing this is, this isa me directive.
Do not pay the ransom. I don't negotiate with
terrorists. If your organization does, then
follow the organization's guidelines.
(28:06):
However, I do not. Well, I followed my
organizations guidelines, but I don't pay ransoms.
This slide right here. This is the flow chart that I
made. It does have my contact
information down at the bottom. The e-mail is going to be the
best way if you have follow up questions.
That phone number, it's mainly used as a voicemail trap for
sales calls or if I need to place calls.
(28:28):
But this, this flow chart is kind of what my brain envisioned
on the paper for other people tobe able to follow.
And I'll give everyone a moment just kind of look over it.
It did end up panning out prettymuch exactly as I I listed it
with little adjustments here andthere.
(28:49):
And then this, there is a a PDF copy of this in the slide
resources and there's also an editable copy.
I did make it in Visio. So it's a VBDSX file, I think.
I think that's what the that file is.
That way you can just kind of take this and run with it as a
template. Obviously I'm not going to be
your incident commander, so you'll want to change that
information if you do recycle it.
(29:12):
Some of the things that we are are proud of, like the reason
that we treat this as a success,the reason we're willing to give
a presentation about this is because so much went well.
You know, someone found a compromise account and got in.
Yes, that is a bad thing. But everything around that is
pretty strong. Brandon had already prepped our
leadership about it. He had a CIRP, he had the team
(29:34):
in place. He didn't get flustered.
He he followed the process. There were backups, enough
backups in place even though onegot compromised.
There were some things that werenot 100% perfect about it.
He had multiple backups and so we were able to restore really
quickly. All essential services were
backup by the very next day, which is amazing.
(29:55):
Great. I think I just said all of those
low recovery costs. OK.
So one really key part of this is that Brandon again has the
knowledge and expertise because he has been in the industry long
enough that he we didn't have tooutsource a lot of the things he
keeps saying that, you know, he was able to act as the the
response command of this. That keeps costs really low.
And so when we debriefed with our vendors and the support that
(30:17):
we had afterwards, that was something that they definitely
noted. They're like, they didn't have
to provide someone else to be that link between everything
because he was able to be that. So sorry, everyone, we aren't
sharing, but Brandon was a really valuable part of that.
I mean, that would be part of our conclusion is like, what do
you do if you don't have a Brandon?
Brandon, do you feel stable enough?
Do you Anything else you want toadd that I didn't cover from
(30:40):
those? No, not for you.
You hit it all on the head. All right.
So for sure, though, yeah, yeah,yeah, I got this one.
I think I thought audios, the APstuff figured out.
All right, So what do we improveafter obviously our backup in an
account auditing, There was a tool I use, and I have to say it
slowly because it definitely sounded really funny when I said
it fast. It's called Ping Castle.
(31:03):
I'll put that in the chat as well.
That's a tool I used. It's a open source tool just to
kind of audit my Active Directory environment.
Puts a whole bunch of different information, but I was able to
see which service accounts I had, which ones had permissions
to wear it was it was great. We also have a different backup
(31:24):
solution. It's not that our the backup
solution time was a bad one. The product itself is good, but
it required too much work on ourend for the size of our team.
So we went with something that'sa little bit more like hybrid
cloud managed. We also went through and cut
down a lot of our service accounts and scaled the back,
made sure that they, if they didn't need to be logged into,
(31:48):
they weren't able to be logged into.
None of them have VPN account orcredentials.
All of them are locked behind MFA.
All of our servers are behind MFA, you know, staff accounts,
all that good stuff. So we, we went through and
tidied up all of our account stuff.
We refined our cyber incident response plan because you always
(32:08):
have to keep refining it. It's not something that you just
make once and you just let it sit for 27 years.
It's almost like every other month you got to be looking at
being like, oh, now we got to kind of modify this.
We got to change this because technology is changing.
So make that a living document. We went through and upgraded our
firewall. We got a brand new firewall,
completely different vendor. Again, not because the product
(32:29):
we had was a bad one, but super outdated.
We wanted something a little bitmore modern and could kind of
work with the setup that we weregoing with because we were at
the time and still are making some back end changes on our
infrastructure itself to kind ofmake things more redundant and
just flow a lot better and be able to handle everything
realistically because we want togive the students the best
(32:49):
experience possible. So we added in a security vendor
to augment our team sock as a service.
They've been great with us. They went through in, they added
in a bunch of different rules and stuff into our crowd strike
environment. They help make sure that you
know, it's getting what it needsto and then they'll poke in when
alerts are kind of coming through and verify if it's a
(33:10):
false positive or not. It's, it's been great.
And then we'll, we'll see stuff and ask them questions and we'll
kind of like jump in, go back and forth.
Already mentioned the removing interactive logins.
We did that through group policy, by the way, on the
Windows side for our service accounts.
And again, if, if you have questions on any of the stuff I
mentioned, my e-mail is available.
(33:33):
We also went through and patchedall the stuff.
We have Tanium for patch management, which I don't think
I mentioned this, yes, but yet. But the Arizona Department of
Homeland Security has a program called the Cyber Readiness
Program where they provide a bunch of security tools at no
cost to state and local education entities.
(33:55):
So we we have some products through them.
So we're getting our Crowdstrike, Titanium and Duo
MFA through a CDHS. It's a great program.
I love it. Otherwise we, we would be not as
far along in our network stuff and security kind of bolstering
as we are now. Next slide.
Ah, yes, so, so when we before we had actually presented this
(34:20):
time at or this presentation, wehad another attempted incident
where they had gotten in throughthis was through another
account. And we're still in the process
of adding VPN into the M or intothe firewall.
Cause the firewall was just put in place and we're still kind of
building everything back out. So they came in through the VPN,
(34:43):
but they were in for less than 15 minutes.
Crowd strike started popping some alerts and blocking stuff.
And then we actually saw that VPN connection in the, the new
firewall and we saw that it justtimed out and there was no
reconnections because we were, we were monitoring for
reconnections on that account. But they, they gave up.
(35:04):
They saw crash strike was on. There was like, Nope, we're not
doing this. So we did go through and, you
know, change that password and everything like that to make
sure that it got under the proper MFA.
But yeah, that one, that one wasjust to show that, hey, what we
did helped. And we definitely did bring this
up to cabinet. We said, hey, look from that
(35:25):
incident that happened, here's you know what we've implemented
and changed, here's what just recently happened and here's how
far they didn't get, which was amazing all.
Right. And this is my cue to take back
over. So you know, in the in this live
session that we gave some chanceto like discuss in groups, talk
to people around you, etcetera. But I hope that this, like our
(35:48):
walkthrough of both what happened and what we did before
and after that made it valuable and something that we get to
brag about has given some information, some ideas that you
could be able to use with your teams.
So these are some questions thatthat should be useful for you to
think about if as you as you consider preparing or auditing
(36:08):
or responding your own cyber incident response plan.
So what would you want to do? These are a lot of the things
that are the practices that we either have or learned or
improved. Make sure that you always know
which accounts exist, what they can access, trim off anything
that is not needed. It is not the most fun task, but
(36:30):
it is one of the most valuable ones.
You know, showing up all of the little the cracks in the wall
that keep the bad guys from getting in.
We really are are grateful that we did that leadership
presentation to let them know that it's not if, but when so
that they weren't startled when it happened.
You can use us as an example of folks who like we, we tried and
(36:51):
we prepared. It still happened to us.
There's nothing you can do the the bad guys are going to find a
way of some sort to try to do something.
So justify at least like a a good 15 minute presentation on
the state of cyber across the country.
Education is a major place that is targeted by these threat
actors so that they are aware. So it's not the first time
(37:14):
they've heard about it when something occurs to you and then
yes, have a plan. Steal ours.
Steal Brandon's as a start for where you can go from there.
And if you don't have a Brandon I again, we will not share him.
He's stuck here. But there is there are ways that
you can address this without having to hire like if you can't
(37:35):
find someone with this expertisewho wants to work in education,
there are hopefully partnershipswith your state and local
governments. Federal government has things
that are available to you to be able to protects the data of our
students. It's a thing that matters.
Yeah. I mean, getting an MFA in place
everywhere you possibly can is one of the most valuable things.
(37:57):
Brandon, do you have anything tothrow of like funding wise?
Like what can you do if we have limited funds and we don't have
a brand and like what would yourwhat do you got?
I mean, the, the difficult part is not having just that kind of
like just the, the cyber knowledge of just all the
(38:17):
different possibilities of what is and, and isn't, but have a
plan. You can find a whole bunch of
different resources online. And I, I should probably just
make like a little quick, Quick guide of like, Hey, what should
you be checking to make sure your stuff is good?
But the biggest things, make sure your backups are good and
that there are backing up and that you have an immutable
(38:38):
backup. I know there's a, there's new
terminology going around that's better than mutable.
I haven't committed to memory yet, but there are better ways
to do it. But make sure your backups are
good. Make sure your service accounts
are locked up and that your accounts have MFA.
Your servers have MFA, right? And if it, if it is something
where it's where it's like a, a cost thing, make sure your
critical servers have MFA and your critical service accounts
(39:01):
or any accounts that have admin privileges, right?
Because you can look at trying to do, you know, scaled stuff.
It would be better if to have everything with MFA.
But if you're on a budget constraint, you do need to still
try to make some steps forward to secure your stuff and then
have a plan. I mean, the other big thing is
just having a plan and knowing what to do when everything kicks
(39:21):
off. You can use that flow chart if
you want. You don't have to reinvent the
wheel because there's 27,000 people random number that have
made the same presentation that we've given, I'm sure.
But there's, there's a lot of resources.
You don't have to reinvent the wheel, but you can take stuff
and tailor it to your organization.
(39:41):
But you definitely want to have a plan.
You always need to have a plan at the bare minimum.
And thanks for sharing that linkin the in the chat to Kosan
Kosan's website about cybersecurity resources because
free resources, a great robust network of folks who can
contribute some to that. So that is another next step
(40:03):
after this. OK, we got to get our house in
order. Let's start working through some
of this last slide. We'll you know.
We will not walk through this, but here's six different things
to do, some sort of audit, like,all right, over the next six
months, what is one or two stepswe can do to make every single
one of these just one notch better than they were at this
(40:23):
moment, right? This can be some sort of
catalyst, some sort of call to action for you and your team to
say, all right, we don't have todo everything.
We can't hire a Brandon, but let's make this.
Let's have a CIRP, let's add, let's audit our backups and make
sure it touches everything or make sure, you know, make a
third backup. So if you know you're hot, warm,
cold, like do we have all of these?
(40:44):
So hopefully that can be something that you can take some
good action steps on. If you do have some questions,
you can throw them in the chat. It's been available the whole
time. Here is our final contact us
slide. That QR code and the Bitly there
will get you access to the resources that we've provided
(41:05):
the the sample communication that we sent out to the
community, the incident responseplan, etcetera.
And I wouldn't have a lot to addif you emailed me, but I bet
Brandon would. So yeah.
Yeah, feel free to e-mail me with any other additional
questions. And there was, so I tried to
remember at the, at the end of our panel at the Coast in
(41:27):
25/20/25 in Seattle, there was abunch of great questions.
And unfortunately, I can't remember all of the, the
questions that we had. It was great back and forth, but
there was one that seemed to be a recurring topic and it was how
to get buy in from, you know, your executives and everything.
And that one is pretty difficult, but there's some
(41:48):
things that you can do to kind of help.
You don't have to be a people person either.
You just need to be able to speak in their language.
So one of the OR a couple of things you can do to help get
yourself by and as you're kind of developing this plan, right,
you want to put in, in the business terms.
So you want to put in terms of money, dollars and impact,
right? How is this going to impact
(42:09):
either funding budgets, wallets,you know, the money stuff,
right? Or how's it going to impact the
service that you provide? And if you're working in
education, the service you provide is education to the
students, right? So how is that going to impact
them, right? Well, if the network's down for
an entire day, that means that'san entire day that the students
don't have access to the learning resources, right?
(42:31):
So you have to, you have to put it in terms that they're going
to understand. That's, that's the, that's step
#1 #2 have a plan ready before you get the buy in right.
So if you're going to say, hey, listen, there's these bad guys
that are going to, they're always trying to get into our
network. Here's some stuff that happens
from our logs and then you just leave with that.
You're probably not going to getthe money you need.
(42:52):
But if you're like, hey, listen,here's some trends that we see.
Here's how it impacts. Here's how we're here's some
things that we're doing now already, right, to help kind of
prevent. Here's some things we want to
do. Here's some, you know, do like a
short term, like, you know, like3 to 5, you're kind of like
window, right view and say theseare things we want to do.
However, we need funding to get some of these.
(43:13):
And here's how each different thing is going to, you know,
positively impact or negatively impact if you don't get the fuck
right. Put it in terms that they can
understand so they can visualizewhat actually happens.
Because a big time or a big problem is with that translation
from technical to non-technical is non-technical.
Can't necessarily like envision all the technical stuff like we
(43:35):
can just because, you know, we've been working in the fields
for so long. We kind of just we know things
they don't know those things that we do.
So we need to put that in their terms, show cost analysis.
If you're able to get a cost analysis going, if you even run
over to your business departmentand say, hey, listen, can you
help me make a cost analysis forthis?
I'm trying to like, you know, get some funding, right?
(43:57):
I am sure that that someone in that business office will be
willing to help you create something like that, especially
if you show interest in that, like their job about it because
people want to talk about their job, right?
So kind of just build that relationship.
You'll be able to start getting a little bit more buy in from
just, you know, people in the inthe departments as well.
(44:18):
That does go a long way. Another thing, transparency,
keep your staff informed. You don't have to tell them all
the nitty gritty. You don't have to tell them all
The Dirty details, but give them, just clue them into what's
going on. If there's network issues going
on, just be like, hey, we're experiencing some network
issues, blah, blah, blah. Over time that does have a very
(44:38):
positive effect because I've been, I started doing that when
I was a site technician at the campus I was at and I still do
it now. It's, it's a little bit
different how I do it now. It's not just sending out all,
all staff emails on blast. We're, we're going into a more
newsletter kind of format just so that way we reduce the amount
of scrolling the staff has to do.
We can keep it like really shortand condensed, but we just keep
(44:59):
them in the loop on what's goingon and when staff has left our
organization. I, I haven't personally heard
it, but my colleagues have wherethey'll be chatting with former
staff members and like, yeah, it's like night and day with the
IT department in terms of just knowing what's going on because
they would go from us to anotherdepartment that doesn't keep
(45:21):
their people in the loop. So just that little bit of
transparency goes a very long way in building that trust
between your staff and your higher ups.
Yes, that's that's about all I have for the the buy in off the
top of my head. Yeah, if there's any other
questions definitely drop them in chat.
If not, y'all have a wonderful, wonderful day there.
(45:43):
You go. Thanks everyone for
participating. Get ready to connect, learn and
lead at Kosin 2026, happening atthis April in Chicago for the
future of education through the lenses of AI, cyber security,
(46:04):
emerging technologies and more. You'll hear from visionary
speakers, dive into hands on sessions, and discover solutions
that move learning forward. All all while networking with
the brightest minds shaping K12 at Tech today.
Don't miss your chance to be a part of the conversation driving
human leadership in a world of AI.
Register now and say with early word rates, visit cosin.org.
(46:28):
That's cosn.org.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Ruthie's Table 4
For more than 30 years The River Cafe in London, has been the home-from-home of artists, architects, designers, actors, collectors, writers, activists, and politicians. Michael Caine, Glenn Close, JJ Abrams, Steve McQueen, Victoria and David Beckham, and Lily Allen, are just some of the people who love to call The River Cafe home. On River Cafe Table 4, Rogers sits down with her customers—who have become friends—to talk about food memories. Table 4 explores how food impacts every aspect of our lives. “Foods is politics, food is cultural, food is how you express love, food is about your heritage, it defines who you and who you want to be,” says Rogers. Each week, Rogers invites her guest to reminisce about family suppers and first dates, what they cook, how they eat when performing, the restaurants they choose, and what food they seek when they need comfort. And to punctuate each episode of Table 4, guests such as Ralph Fiennes, Emily Blunt, and Alfonso Cuarón, read their favourite recipe from one of the best-selling River Cafe cookbooks. Table 4 itself, is situated near The River Cafe’s open kitchen, close to the bright pink wood-fired oven and next to the glossy yellow pass, where Ruthie oversees the restaurant. You are invited to take a seat at this intimate table and join the conversation. For more information, recipes, and ingredients, go to https://shoptherivercafe.co.uk/ Web: https://rivercafe.co.uk/ Instagram: www.instagram.com/therivercafelondon/ Facebook: https://en-gb.facebook.com/therivercafelondon/ For more podcasts from iHeartRadio, visit the iheartradio app, apple podcasts, or wherever you listen to your favorite shows. Learn more about your ad-choices at https://www.iheartpodcastnetwork.com
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com