June 3, 2025 • 24 mins
In this episode of Tank Talk, Shannon and Cassie dive into the hot topic of AI and environmental compliance. Cassie Kuzis, part of the Integrity Environmental team, joins the podcast for the first time with Shannon to weigh in on AI. With AI tools becoming more common in operations, Shannon decided to run a little experiment: Can ChatGPT write a Spill Prevention, Control, and Countermeasure (SPCC) Plan?
Spoiler alert - it didn’t go well.
Shannon walks us through the results of three AI-generated SPCC plans, revealing the major compliance failures, regulatory inaccuracies, and risky oversights that came with each attempt. From missing signatures and misapplied containment definitions to false statements and oversimplified training requirements, the experiment highlights why environmental plans and permits should always be developed by qualified professionals.
Tune in to hear:
- Why SPCCs are legal documents - and why that matters
- Where AI went wrong (and what it surprisingly got right)
- What you should do when considering tech support for environmental planning
This is a must-listen for facility managers, compliance officers, and anyone curious about the limits of AI in regulated industries.
Additional Resources & AI Prompts from the episode:
https://www.integrity-env.com/i-used-ai-to-write-my-spcc
intro/outro created with GarageBand
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Haley Hall (00:10):
Welcome to Tank Talk with Integrity Environmental.
Join us as we sit down withfounder, principal consultant
and bulk fuel storage expert,Shannon Oelkers, to explore
regulations, safety andessential tips for navigating
the bulk fuel storage industry.
Join us as we explore theunique joys of work and life in
Alaska with industry experts,including our team, vendors we
(00:30):
work with and the companies wesupport.
Shannon Oelkers (00:35):
Well, hello, welcome back to Tank Talk.
This is Shannon Oelkers, yourhost, and today I'm joined by
Cassie Kuzis.
She's a member of our salesteam and her and I today are
going to do something reallyinteresting.
I used AI, Cassie, to write anSPCC plan.
Whoa, I know right.
(00:56):
Like, should we do that?
I don't even know.
So before we begin, Cassie, Ido want to take a minute to just
highlight something aboutIntegrity and AI.
Before we continue any further,at Integrity, we want to make
it clear that, due to the highlyregulated and legally complex
nature of environmental planningand permitting, we never use
(01:17):
artificial intelligence todevelop environmental plans or
permits.
All of our work in this area iscarried out by experienced
environmental professionals whounderstand the nuances of
regulations and the uniquerequirements of each facility.
I want to reassure everybodywe're committed to ensuring that
every project we undertakemeets the highest legal and
ethical standards, and we remaindedicated to providing expert,
(01:38):
human-driven solutions to ourclients.
So there's all the legal stuffout of there, but I still used
AI to do something fun.
So I used ChatGPT that's the AIthat I have access to for free
to write an SPCC.
Cassie Kuzis (01:57):
What inspired you to do that?
Shannon Oelkers (01:59):
Well, it was part natural curiosity just to
see what it would spit out.
But, Cassie, I was alsointerested because of how we see
clients beginning to use AI tosupport operations.
You know, I had a feeling thatsomeone somewhere was going to
have the bright idea of havingAI help them draft up an SPCC.
Cassie Kuzis (02:19):
Oh yeah, that seems very likely.
So what happened?
What did you learn?
Shannon Oelkers (02:24):
Well, I learned a whole lot, but before we dive
into the findings, though, Iwanted to provide some
background information that Ithink is relevant to this, which
is part of what I learned, andeverybody who listens to this
podcast knows how much I love mesome context.
So first I want everyone toremember and we've talked about
this before in some of our otherpodcast topics but at the
fundamental level, a spillprevention, control and
(02:46):
countermeasure plan, or SPCCplan, is a legal document.
It's part of your agreementwith the federal government to
store fuel in bulk and meet theregulations of that 40 CFR 112.
So all of the warnings thatapply to legal documents will
also apply to environmentalplanning and permitting.
If there is a legal complianceor enforcement arm to a
(03:07):
regulatory agency, any plan orpermit you have through that is
going to be a legal document.
Cassie Kuzis (03:13):
Gosh, it seems like it might be a bad idea to
use AI for legal documents.
Shannon Oelkers (03:19):
You would be correct.
Yes, for one thing, the AImodel is really dependent,
Cassie, on the information thathas already been given to it.
I have noticed that, especiallyfor things like environmental
permitting, there's a lot ofmisinformation on the internet
or overly simplified information.
There's not a whole lot ofinformation about our industry,
operations and infrastructurethat's available to, like most
(03:41):
AI models, in a context thatwould make sense for a planner
permit.
I'm sure there's some verysophisticated companies out
there that have paid for theirown learning management systems
and AI and they're training themon things like this.
However, even if the AI had goodinputs, you have to have well
phrased prompts, and if youdon't have well phrased prompts,
(04:03):
your AI can return nonsenseanswers with a really high
degree of confidence, based onthe inaccurate data that the AI
has access to.
So the second half of thisequation is, if you don't know
exactly what you're asking for,your prompts give you these
really murky answers.
So most of the people we dealwith you know they understand
and know what an SPCC is, butthey don't necessarily
(04:23):
understand many otherenvironmental permitting
regulations that are out there,like NEPA or stormwater.
And if you ask a pretty vaguequestion like write a spill plan
for my facility, which sounds awhole lot like.
Write an SPCC for my facility.
Your learning management systemor AI will return a compliant
plan for something completelydifferent, like hazardous
(04:44):
material spills instead of bulkfuel oil or something like that.
It might ignore fuels entirely.
We call it the garbage in,garbage out problem.
If you're not asking the rightquestions in the right way,
you're not going to get theright things coming out of it.
Cassie Kuzis (04:58):
So that makes sense.
Don't use AI for complex legaldocuments.
Shannon Oelkers (05:03):
I feel like there's a lot of content on the
internet already reinforcingthat, but I did want to cover
those basics because that's thefirst problem with using AI for
environmental permitting is thatit's legal documents in
addition to environmentalpermitting.
So there's some reasons why weshouldn't do that.
But now that we've coveredthose basics, I still decided to
do it anyways, and here's why Iwanted to see what it put out
(05:27):
and I also wanted to see whatthe quality of the output was
and then kind of discuss it hereon the podcast so we could all
learn from it.
Cassie Kuzis (05:34):
Shannon, what did you start off with?
Shannon Oelkers (05:37):
Well, when I went on to AI, I originally used
one prompt and we did use chat,gpt, but I felt like the one
prompt wasn't robust enough, soI actually went ahead and did
two more.
So I have three prompts and Iwill include all of the nominees
air quotes here for those ofyou who can't see me for the
plans that these three promptscame out with.
And then I added some super funcommentary in the Word
(06:00):
documents, just so you can.
You know, you can judge alongwith me as I went through the
plans, and those will beincluded in our show notes if
you want to open them up and seewhat came out yourself, just
out of curiosity.
So here's the three prompts Iuse Cassie, the first one, very
basic write a spill prevention,control and countermeasure
parentheses SPCC plan inaccordance with 40 CFR 112.
(06:22):
Then my second prompt I waslike okay, that was very general
, let me, I'm going to go alittle deeper.
So the second prompt I saidwrite a spill prevention,
control and countermeasures planin accordance with 40 CFR 112
for a facility with one 4,000gallon tank storing gasoline and
one 6,000 gallon tank storingdiesel, plus four drums, 55
(06:43):
gallons in size, storing motoroil, 5w30 motor oil.
And then my last prompt I kindof wanted to see what happened
when I took the tanks out, so Isaid write a spill prevention,
control and countermeasures planfor a facility with 20 isotopes
that are 330 gallons eachstored in a warehouse.
Cassie Kuzis (07:00):
Okay, so we've covered a few different options.
How did they show up?
Shannon Oelkers (07:11):
Overall for all three plans that the chat GPT
generated.
It wrote pretty much what Iexpected.
It spit out this like barebones skeleton report with a lot
of direction to the writer tofill in the blanks.
Right, but it did have somesignificant errors and
assumptions and there were a fewsurprises where it did better
than I thought.
Cassie Kuzis (07:26):
What were the significant errors you found?
Shannon Oelkers (07:29):
Well, I'll go through what all three versions
missed.
I think that's the key part.
So some things were captured insome of the responses and some
were missed in others, but allof the versions missed.
They did not have a completerequired commitments and
signatures section and I'll gointo these in more detail.
(07:49):
But there are certain thingsthat the regulation says have to
be signed and statements thathave to be made as part of the
SPCC plan.
None of these three plans had acomplete and compliant section
there.
None of the plans captured theprincipal engineer review
correctly for SPCCs.
All three of them were missingrequired content.
It wasn't all the same contentthat was missing, which I find
(08:12):
really interesting, but all ofthem showed that there was a
couple sections missing thatwere supposed to be there and it
was independent of thecontainers, like they all should
have had certain sections,regardless of what type of fuel
in the containers they werestoring it in.
But it was still missed.
The other thing all three ofthese missed was referenced
regulations and engineeringstandards.
(08:33):
So in a regulation it'll saythings like inspect the tank to
best engineering practices orrecognized engineering standards
or, in accordance with 40 CFR63, ensure the tank venting is
appropriate, all right.
None of those cross-referencedregulations and engineering
standards came across at all.
(08:54):
They were invisible to the AI.
If it was cross-referenced orreferenced to somewhere else, it
just didn't follow that trailat all or even make a statement
about it.
It's just invisible.
There was also some falsestatements in the content and
all three of the plans and youcan go to my commentary in each
of the plans to see.
But they made a lot of sort oftruthful sounding statements but
(09:14):
not really.
And then there was definitelysome wrong context for certain
words, like the word containmentfor spill prevention or sorry,
spcc plans has a very specificdefinition in the plan but AI
was applying the more generalacross the whole internet
approach definition ofcontainment and same with
drainage and a few other wordswhere you could just tell, by
(09:37):
the way the AI was writing itthat it was not using the
specific definition within 40CFR 112.
It was looking at it from whenpeople on the internet write the
word drainage what they mean.
Just a couple other things.
None of them addresssubstantial harm determination
and that may be because it'sactually included in a different
set of regs, but every SPCC hasto include it to show that they
(09:58):
do or do not need a facilityresponse plan to go with the
SPCC.
So that was a little funky.
I was like it needs to be inhere, but it's not.
And then the last thing Inoticed amongst all three of
these was that many of thesections like employee training
and figures they were sogenerally listed that you
couldn't actually create acompliant or adequate figure or
(10:20):
a training program out of whatthey spit out.
Cassie Kuzis (10:24):
Gosh.
Well, that's quite a fewproblems.
Shannon Oelkers (10:27):
It really really is.
Stormwater compliance on a bulkfuel site isn't just a box to
check, it's critical.
Integrity's CGP SWPPP servicesmake it simple.
They deliver fast, professionalplans designed for real-world
use so you can stay compliantwithout any headaches.
Our team has decades ofexperience, works directly on
(10:49):
job sites and knows exactly whatit takes to keep your project
running smoothly.
Plus, every bid includes a freehour of consulting because when
issues come up, you're going toneed those answers and fast
Stay ahead of compliance.
Get a bid today.
Send us an email at info atintegrity-envcom.
Integrity stormwater solutionsyou can trust.
If you're game Cassie, I candig a little deeper on these and
(11:14):
just sort of go through myfindings.
Cassie Kuzis (11:17):
Yeah let's dig in Okay.
Shannon Oelkers (11:20):
So the first one was like the inaccuracy in
signing and commitments.
They basically had aninconsistent listing of the
responsible person and then theyhad varying levels of
committing resources andattesting to the accuracy of the
content.
Prompt one that plan that wasgenerated didn't have any place
for the owner or representativeto sign.
Prompt two had the owner andrep signature but not the
(11:43):
principal engineer.
And then the prompt threeversion had it had a prepared by
, reviewed by block, but it wasmissing the attestation for
accuracy and commitment ofresources altogether.
So it was sort of this weirdlike parts of it came through
but not all of it for any ofthem.
For the principal engineeringreview requirement those of you
(12:04):
familiar with SPCCs willremember that 40 CFR 112.3 has
specific requirements for PEreview and there's exemptions
for facilities under 10,000gallons where you can do self
certification.
But self-certification, youknow, of course, only applies if
you're fully compliant.
If there's anything that'snon-compliant and you have to
have deviations for any reason,then the PE has got to get
(12:26):
involved.
So it's a little complicatedbut none of the plans even
addressed any of that.
It just kind of AI got it wrongevery which way.
So for prompt one, ai statedthat the PE review was required,
but it didn't touch on volumethreshold exclusions and that
prompt was pretty vague, itdidn't include any volume
(12:46):
information.
So I would have expected it tosay PE review, you know, applies
if over 10,000 gallons or theseconditions exist, because
that's how the reg is writ.
Didn't show up in the AI Prompt.
Two did need a PE stamp.
It exceeded the 10,000 galloncriteria but it wasn't addressed
at all in that prompt.
And then the third one it didnot need a PE stamp but it
didn't include language thatwould identify the facility as a
(13:07):
tier one and self-certifying,which is the alternate to the P
stamp.
So all of these plans that weregenerated, they just didn't
really get the PE part correct,which you know.
Fair enough, that's one of themore complicated pieces For not
including the required content.
All three of these plans werejust missing whole sections on
like facility security, employeetraining, spill prevention.
(13:30):
What exactly got skipped varieda little bit between the
different prompts, but not oneof them had all of the required
sections.
There's also referenceregulations and engineering
standards that were missed.
The biggest one that we saw wasthat if you have a bulk fuel
tank under an SPCC plan, there'srequired third-party
inspections under STI SP001 orAPI 653 for the different kinds
(13:54):
of tanks and at differentvolumes of tanks.
None of the plans touched onthat at all.
It was invisible, not sure whatelse to say, but if you wrote
it you would have no idea thatyou needed to have your tanks
inspected in accordance withthose two engineering standards.
And this one's interesting,cassie they had some false
statements in there, like AI wasparaphrasing or generalizing so
much that, like in the first,the prompt one response plan, it
(14:17):
said that it required 110% ofcontainer volume for sized
secondary containment.
And that's just not true.
It's a good general rule ofthumb, but if a secondary
containment is not protectedfrom the weather, it's got to
include an allowance for the 24hour 25 year rainfall number and
that comes from fire code.
But it's also listed in someguidance that the EPA has
(14:39):
published since the SPCCregulation came out, and so this
kind of goes back to thatfailure of referencing
regulations and engineeringstandards, like it's making
false statements because it isunable to cross-reference all
these things and make theaccurate statement.
It's just a little weird.
You see it and you're like it'salmost right but it's not.
And when it comes to compliance.
(15:01):
That's still a compliance issue.
And then, definitely throughoutall of these plans, we just saw
the wrong context being appliedto really critical words like
containment.
It does not appear to me that AIis actually pulling the
definitions from 40 CFR, 112,the regulation itself it's using
(15:21):
, or, if it is, it's such asmall drop in the pool compared
to the overwhelming evidence onthe wider internet that it just
can't bring it up.
And there may be some promptingthat I could do.
I'm sure somebody listening tothis who's an AI expert would
like.
If only you had done thisprompt, it would have been
better.
But I was trying to put myselfinto like the average regular
person that was trying to writean SPCC for their facility and
(15:44):
then some of the storagecontainment volumes were just
mixed up again.
A good example of this is likewhen we prepare SPCC plans,
there's secondary containmentrequirements for all these
different kinds of containers.
Some of those requirements aregeneral, like to collect weeps,
drips and spills, you know, likethe spill bucket at the fill
point of your tank, for example.
(16:05):
But others are like sizedrequirements.
So you have a 330 gallonisotope.
You need to have containmentfor it that is equal to 330
gallons in size, right, plus 10%for precipitation if it's
outside and exposed tostormwater.
Well, depending on thecontainer type and how it's used
and where it's located, you cansort of have all these
(16:26):
different containmentapplications and it can be
really confusing.
So, for example, let's saywe've got a helicopter company
and they want to store 10, 55gallon drums in a remote
location as like a refuelingsupply station.
Right, if you're the helicoptercompany and you're building a
containment area for these 55gallon drums and there's 10 of
(16:47):
them does your secondarycontainment need to be 550
gallons, as big as everythingyou've got, 55 gallons for the
biggest container or just fivegallons to capture like weeps,
drips and leaks?
Cassie Kuzis (17:01):
I'm not sure off the top of my head.
Shannon Oelkers (17:04):
A lot of people , don't?
We get this question a lot.
Actually, under 40 CFR 112, thecorrect answer would be 55
gallons, the volume of thelargest container within the
containment area.
But if you don't have thatcontext or definition, using AI
to create your plan, we'llabsolutely spit out a wrong
answer.
In the prompt number two, wherewe had two tanks, it said that
(17:25):
the tanks had to havecontainment that was equal to,
you know, 4000 gallons plus 10%,right, 110%.
But it also said the tanks weredouble walled.
So if you were writing thisSPCC plan and getting ready to,
you know, buy a double walledtank, for example, you're
doubling up on the containmentand that's an expense you
wouldn't have needed to dobecause you've got double walled
(17:45):
tanks, which is your 110%containment already.
So that kind of complexitytheir AI is nowhere near
touching.
There's also a couple ofexamples where they listed the
broad line items for things like.
It would say things likeemployee training will be
provided for, spill response, orfigures showing the facility
layout will be included in anappendix, but that leaves out
(18:07):
any specifics for what thecontent of those things needs to
be and the SPCC is prettyspecific about.
You need to have the followingthings in your figures.
You need to have the followingpieces of your training program,
and so I feel like the spillresponse actions were somewhat
captured in the plans better,but spill prevention, like
transfer operations andmaintenance, and they were all
(18:29):
left out of all three of them.
They just didn't reference themat all.
They just didn't reference themat all.
I feel like, after going throughmultiple reviews with the EPA,
with our SPCC plans in general,these overgeneralized language
that AI is producing is just notgoing to be compliant in the
eyes of the EPA because itdoesn't address specific nuances
(18:50):
of each of the regulatoryrequirements.
And I guess a good example ofthis it's a difference between
check the truck for leaks andcheck the lowermost drains and
valves for leaks prior tofueling and prior to departure.
One of those is based on aregulatory language, the other
one is a overgeneralizedstatement and a lot of people
are like, oh yeah, I did, Ichecked the truck for leaks, but
(19:12):
the EPA, with 100% authority Ican say, will assign a violation
if you don't include thelowermost drains and valves in
your checklist and can show thatthat's part of your training to
the employee.
Cassie Kuzis (19:24):
Ah, so there's just a lot of nuance there, yeah
, and that's just like one tinypart Did any big surprises come
up?
Shannon Oelkers (19:32):
Yeah, there was a couple.
For whatever reason, AI gotportable drums and containers
pretty well.
I don't know if that's due tomore content about drum storage
and warehousing being availableon the internet or what, but it
got a lot of the portable drumsand container stuff pretty
accurate and pretty thoroughly.
Labeling and signage made anappearance on the third prompt.
(19:52):
It did not make an appearancein the first two, which both had
portable containers and tanks.
So I don't know why, but I didnotice that labeling and signage
finally made it on there on thethird time around.
It also got spill reportingreally well.
Again, I think likely becausethere's so much correct
information on the internetabout that specific task already
(20:12):
, because lots of people havehow to notify, how to do spill
notifications posted, you know,to the internet.
Cassie Kuzis (20:19):
Wow, that is really interesting, but it still
seems like for now I shouldprobably leave AI out of SPCCs.
Shannon Oelkers (20:26):
I really agree with that.
Honestly, I think if you wantedto create your own SPCC, just
reading the regulation line byline and responding to each
portion of that regulation wouldgive you much better results
than AI.
But in reality, I think mostcommercial facilities need a
professional plan writer, notjust reading it line by line,
because you need somebody whounderstands the regulations, the
(20:49):
context, your industry andoperations and that's really
going to turn into the bestoutcome for your facility
overall.
Cassie Kuzis (20:57):
Professionals, like I said, integrity.
Shannon Oelkers (21:00):
Of course you can't see me through the radio,
but I'm blushing modestly.
Yes, Professionals likeIntegrity or engineering groups,
you know you know youroperations best.
The regulations and theengineering standards and the
fire code all play into writingan SPCC plan.
I don't think it's typical formost people who are focused on
the main job task of storingfuel to also have a good grasp
(21:22):
of the fire code and engineeringstandards and inspection
standards, and so all of thatstuff feeds into.
If you have a professional thatcan help you and they're
working with you, you'll get thebest of both worlds.
You get your land-basedknowledge from being right there
on the ground doing the job,combined with their professional
knowledge about all of thesedifferent regulations and
standards.
Cassie Kuzis (21:41):
Well, do you think we should be worried about AI
in the future?
Will it ever be able to writethese plans?
Shannon Oelkers (21:47):
I think our jobs are safe from AI for now
and I have a feeling maybeforever, because facilities are
pretty straightforward when theyare built, but facilities
change as they age and I don'tknow that AI can overcome
facility specific content, likeit's got to overcome the as
(22:09):
designed right and then linkingthat at through time to the as
observed, and so the changes afacility goes through in 15
years, even if you fed the AI,your engineer design work work,
the tank farm is going to bedifferent and there's all these
like operational things abouthow they transfer fuel and the
ways that they do things andwhen and how they use drip pan.
(22:31):
You know, like there's just alot of things that I think that
human connector is always goingto need to do, because you know,
we know this from writing planswhat you write is never what
you find in reality, and so youneed to constantly connect that
you know, connect betweenreality and ideal, and I don't
think AI can do that without ahuman person.
Cassie Kuzis (22:50):
Well, that's a relief.
Shannon Oelkers (22:52):
Yes, that kind of wraps up what I found out
about AI.
We'll get those show notesposted.
Cassie Kuzis (23:01):
Yeah, thanks for doing that.
I'm sure a lot of people willbe interested in your results.
Shannon Oelkers (23:06):
Yeah, it should be fun to see what else people
see.
If you end up having commentsor you have some ideas about
this topic, we would love tohear them.
Go ahead and email us at infoat integrity dash envcom.
Or if you have ideas for futurepodcasts, I would really enjoy
hearing those.
We've had quite a few listenerideas in the past, so if you've
got one, let me have it.
Cassie Kuzis (23:27):
Awesome.
Well, thanks for that.
Shannon Oelkers (23:30):
Hi there, this is Shannon Oelkers and, as the
owner of Integrity Environmental, I wanted to take a minute here
at the end of the podcast tomake sure that you knew the
following this podcast is forinformational purposes only and
(23:54):
should not be considered legalor regulatory.
Thank you very much forlistening, and the views
expressed in this podcast maynot be those of the host that
would be me or IntegrityEnvironmental.
Thank you very much forlistening and if you do need
professional regulatory advice,we'd be happy to help you as
part of our consulting services.
Welcome to Tank Talk with Integrity Environmental.
Join us as we sit down withfounder, principal consultant
and bulk fuel storage expert,Shannon Oelkers, to explore
regulations, safety andessential tips for navigating
the bulk fuel storage industry.
Join us as we explore theunique joys of work and life in
Alaska with industry experts,including our team, vendors we
(00:30):
work with and the companies wesupport.
Shannon Oelkers (00:35):
Well, hello, welcome back to Tank Talk.
This is Shannon Oelkers, yourhost, and today I'm joined by
Cassie Kuzis.
She's a member of our salesteam and her and I today are
going to do something reallyinteresting.
I used AI, Cassie, to write anSPCC plan.
Whoa, I know right.
(00:56):
Like, should we do that?
I don't even know.
So before we begin, Cassie, Ido want to take a minute to just
highlight something aboutIntegrity and AI.
Before we continue any further,at Integrity, we want to make
it clear that, due to the highlyregulated and legally complex
nature of environmental planningand permitting, we never use
(01:17):
artificial intelligence todevelop environmental plans or
permits.
All of our work in this area iscarried out by experienced
environmental professionals whounderstand the nuances of
regulations and the uniquerequirements of each facility.
I want to reassure everybodywe're committed to ensuring that
every project we undertakemeets the highest legal and
ethical standards, and we remaindedicated to providing expert,
(01:38):
human-driven solutions to ourclients.
So there's all the legal stuffout of there, but I still used
AI to do something fun.
So I used ChatGPT that's the AIthat I have access to for free
to write an SPCC.
Cassie Kuzis (01:57):
What inspired you to do that?
Shannon Oelkers (01:59):
Well, it was part natural curiosity just to
see what it would spit out.
But, Cassie, I was alsointerested because of how we see
clients beginning to use AI tosupport operations.
You know, I had a feeling thatsomeone somewhere was going to
have the bright idea of havingAI help them draft up an SPCC.
Cassie Kuzis (02:19):
Oh yeah, that seems very likely.
So what happened?
What did you learn?
Shannon Oelkers (02:24):
Well, I learned a whole lot, but before we dive
into the findings, though, Iwanted to provide some
background information that Ithink is relevant to this, which
is part of what I learned, andeverybody who listens to this
podcast knows how much I love mesome context.
So first I want everyone toremember and we've talked about
this before in some of our otherpodcast topics but at the
fundamental level, a spillprevention, control and
(02:46):
countermeasure plan, or SPCCplan, is a legal document.
It's part of your agreementwith the federal government to
store fuel in bulk and meet theregulations of that 40 CFR 112.
So all of the warnings thatapply to legal documents will
also apply to environmentalplanning and permitting.
If there is a legal complianceor enforcement arm to a
(03:07):
regulatory agency, any plan orpermit you have through that is
going to be a legal document.
Cassie Kuzis (03:13):
Gosh, it seems like it might be a bad idea to
use AI for legal documents.
Shannon Oelkers (03:19):
You would be correct.
Yes, for one thing, the AImodel is really dependent,
Cassie, on the information thathas already been given to it.
I have noticed that, especiallyfor things like environmental
permitting, there's a lot ofmisinformation on the internet
or overly simplified information.
There's not a whole lot ofinformation about our industry,
operations and infrastructurethat's available to, like most
(03:41):
AI models, in a context thatwould make sense for a planner
permit.
I'm sure there's some verysophisticated companies out
there that have paid for theirown learning management systems
and AI and they're training themon things like this.
However, even if the AI had goodinputs, you have to have well
phrased prompts, and if youdon't have well phrased prompts,
(04:03):
your AI can return nonsenseanswers with a really high
degree of confidence, based onthe inaccurate data that the AI
has access to.
So the second half of thisequation is, if you don't know
exactly what you're asking for,your prompts give you these
really murky answers.
So most of the people we dealwith you know they understand
and know what an SPCC is, butthey don't necessarily
(04:23):
understand many otherenvironmental permitting
regulations that are out there,like NEPA or stormwater.
And if you ask a pretty vaguequestion like write a spill plan
for my facility, which sounds awhole lot like.
Write an SPCC for my facility.
Your learning management systemor AI will return a compliant
plan for something completelydifferent, like hazardous
(04:44):
material spills instead of bulkfuel oil or something like that.
It might ignore fuels entirely.
We call it the garbage in,garbage out problem.
If you're not asking the rightquestions in the right way,
you're not going to get theright things coming out of it.
Cassie Kuzis (04:58):
So that makes sense.
Don't use AI for complex legaldocuments.
Shannon Oelkers (05:03):
I feel like there's a lot of content on the
internet already reinforcingthat, but I did want to cover
those basics because that's thefirst problem with using AI for
environmental permitting is thatit's legal documents in
addition to environmentalpermitting.
So there's some reasons why weshouldn't do that.
But now that we've coveredthose basics, I still decided to
do it anyways, and here's why Iwanted to see what it put out
(05:27):
and I also wanted to see whatthe quality of the output was
and then kind of discuss it hereon the podcast so we could all
learn from it.
Cassie Kuzis (05:34):
Shannon, what did you start off with?
Shannon Oelkers (05:37):
Well, when I went on to AI, I originally used
one prompt and we did use chat,gpt, but I felt like the one
prompt wasn't robust enough, soI actually went ahead and did
two more.
So I have three prompts and Iwill include all of the nominees
air quotes here for those ofyou who can't see me for the
plans that these three promptscame out with.
And then I added some super funcommentary in the Word
(06:00):
documents, just so you can.
You know, you can judge alongwith me as I went through the
plans, and those will beincluded in our show notes if
you want to open them up and seewhat came out yourself, just
out of curiosity.
So here's the three prompts Iuse Cassie, the first one, very
basic write a spill prevention,control and countermeasure
parentheses SPCC plan inaccordance with 40 CFR 112.
(06:22):
Then my second prompt I waslike okay, that was very general
, let me, I'm going to go alittle deeper.
So the second prompt I saidwrite a spill prevention,
control and countermeasures planin accordance with 40 CFR 112
for a facility with one 4,000gallon tank storing gasoline and
one 6,000 gallon tank storingdiesel, plus four drums, 55
(06:43):
gallons in size, storing motoroil, 5w30 motor oil.
And then my last prompt I kindof wanted to see what happened
when I took the tanks out, so Isaid write a spill prevention,
control and countermeasures planfor a facility with 20 isotopes
that are 330 gallons eachstored in a warehouse.
Cassie Kuzis (07:00):
Okay, so we've covered a few different options.
How did they show up?
Shannon Oelkers (07:11):
Overall for all three plans that the chat GPT
generated.
It wrote pretty much what Iexpected.
It spit out this like barebones skeleton report with a lot
of direction to the writer tofill in the blanks.
Right, but it did have somesignificant errors and
assumptions and there were a fewsurprises where it did better
than I thought.
Cassie Kuzis (07:26):
What were the significant errors you found?
Shannon Oelkers (07:29):
Well, I'll go through what all three versions
missed.
I think that's the key part.
So some things were captured insome of the responses and some
were missed in others, but allof the versions missed.
They did not have a completerequired commitments and
signatures section and I'll gointo these in more detail.
(07:49):
But there are certain thingsthat the regulation says have to
be signed and statements thathave to be made as part of the
SPCC plan.
None of these three plans had acomplete and compliant section
there.
None of the plans captured theprincipal engineer review
correctly for SPCCs.
All three of them were missingrequired content.
It wasn't all the same contentthat was missing, which I find
(08:12):
really interesting, but all ofthem showed that there was a
couple sections missing thatwere supposed to be there and it
was independent of thecontainers, like they all should
have had certain sections,regardless of what type of fuel
in the containers they werestoring it in.
But it was still missed.
The other thing all three ofthese missed was referenced
regulations and engineeringstandards.
(08:33):
So in a regulation it'll saythings like inspect the tank to
best engineering practices orrecognized engineering standards
or, in accordance with 40 CFR63, ensure the tank venting is
appropriate, all right.
None of those cross-referencedregulations and engineering
standards came across at all.
(08:54):
They were invisible to the AI.
If it was cross-referenced orreferenced to somewhere else, it
just didn't follow that trailat all or even make a statement
about it.
It's just invisible.
There was also some falsestatements in the content and
all three of the plans and youcan go to my commentary in each
of the plans to see.
But they made a lot of sort oftruthful sounding statements but
(09:14):
not really.
And then there was definitelysome wrong context for certain
words, like the word containmentfor spill prevention or sorry,
spcc plans has a very specificdefinition in the plan but AI
was applying the more generalacross the whole internet
approach definition ofcontainment and same with
drainage and a few other wordswhere you could just tell, by
(09:37):
the way the AI was writing itthat it was not using the
specific definition within 40CFR 112.
It was looking at it from whenpeople on the internet write the
word drainage what they mean.
Just a couple other things.
None of them addresssubstantial harm determination
and that may be because it'sactually included in a different
set of regs, but every SPCC hasto include it to show that they
(09:58):
do or do not need a facilityresponse plan to go with the
SPCC.
So that was a little funky.
I was like it needs to be inhere, but it's not.
And then the last thing Inoticed amongst all three of
these was that many of thesections like employee training
and figures they were sogenerally listed that you
couldn't actually create acompliant or adequate figure or
(10:20):
a training program out of whatthey spit out.
Cassie Kuzis (10:24):
Gosh.
Well, that's quite a fewproblems.
Shannon Oelkers (10:27):
It really really is.
Stormwater compliance on a bulkfuel site isn't just a box to
check, it's critical.
Integrity's CGP SWPPP servicesmake it simple.
They deliver fast, professionalplans designed for real-world
use so you can stay compliantwithout any headaches.
Our team has decades ofexperience, works directly on
(10:49):
job sites and knows exactly whatit takes to keep your project
running smoothly.
Plus, every bid includes a freehour of consulting because when
issues come up, you're going toneed those answers and fast
Stay ahead of compliance.
Get a bid today.
Send us an email at info atintegrity-envcom.
Integrity stormwater solutionsyou can trust.
If you're game Cassie, I candig a little deeper on these and
(11:14):
just sort of go through myfindings.
Cassie Kuzis (11:17):
Yeah let's dig in Okay.
Shannon Oelkers (11:20):
So the first one was like the inaccuracy in
signing and commitments.
They basically had aninconsistent listing of the
responsible person and then theyhad varying levels of
committing resources andattesting to the accuracy of the
content.
Prompt one that plan that wasgenerated didn't have any place
for the owner or representativeto sign.
Prompt two had the owner andrep signature but not the
(11:43):
principal engineer.
And then the prompt threeversion had it had a prepared by
, reviewed by block, but it wasmissing the attestation for
accuracy and commitment ofresources altogether.
So it was sort of this weirdlike parts of it came through
but not all of it for any ofthem.
For the principal engineeringreview requirement those of you
(12:04):
familiar with SPCCs willremember that 40 CFR 112.3 has
specific requirements for PEreview and there's exemptions
for facilities under 10,000gallons where you can do self
certification.
But self-certification, youknow, of course, only applies if
you're fully compliant.
If there's anything that'snon-compliant and you have to
have deviations for any reason,then the PE has got to get
(12:26):
involved.
So it's a little complicatedbut none of the plans even
addressed any of that.
It just kind of AI got it wrongevery which way.
So for prompt one, ai statedthat the PE review was required,
but it didn't touch on volumethreshold exclusions and that
prompt was pretty vague, itdidn't include any volume
(12:46):
information.
So I would have expected it tosay PE review, you know, applies
if over 10,000 gallons or theseconditions exist, because
that's how the reg is writ.
Didn't show up in the AI Prompt.
Two did need a PE stamp.
It exceeded the 10,000 galloncriteria but it wasn't addressed
at all in that prompt.
And then the third one it didnot need a PE stamp but it
didn't include language thatwould identify the facility as a
(13:07):
tier one and self-certifying,which is the alternate to the P
stamp.
So all of these plans that weregenerated, they just didn't
really get the PE part correct,which you know.
Fair enough, that's one of themore complicated pieces For not
including the required content.
All three of these plans werejust missing whole sections on
like facility security, employeetraining, spill prevention.
(13:30):
What exactly got skipped varieda little bit between the
different prompts, but not oneof them had all of the required
sections.
There's also referenceregulations and engineering
standards that were missed.
The biggest one that we saw wasthat if you have a bulk fuel
tank under an SPCC plan, there'srequired third-party
inspections under STI SP001 orAPI 653 for the different kinds
(13:54):
of tanks and at differentvolumes of tanks.
None of the plans touched onthat at all.
It was invisible, not sure whatelse to say, but if you wrote
it you would have no idea thatyou needed to have your tanks
inspected in accordance withthose two engineering standards.
And this one's interesting,cassie they had some false
statements in there, like AI wasparaphrasing or generalizing so
much that, like in the first,the prompt one response plan, it
(14:17):
said that it required 110% ofcontainer volume for sized
secondary containment.
And that's just not true.
It's a good general rule ofthumb, but if a secondary
containment is not protectedfrom the weather, it's got to
include an allowance for the 24hour 25 year rainfall number and
that comes from fire code.
But it's also listed in someguidance that the EPA has
(14:39):
published since the SPCCregulation came out, and so this
kind of goes back to thatfailure of referencing
regulations and engineeringstandards, like it's making
false statements because it isunable to cross-reference all
these things and make theaccurate statement.
It's just a little weird.
You see it and you're like it'salmost right but it's not.
And when it comes to compliance.
(15:01):
That's still a compliance issue.
And then, definitely throughoutall of these plans, we just saw
the wrong context being appliedto really critical words like
containment.
It does not appear to me that AIis actually pulling the
definitions from 40 CFR, 112,the regulation itself it's using
(15:21):
, or, if it is, it's such asmall drop in the pool compared
to the overwhelming evidence onthe wider internet that it just
can't bring it up.
And there may be some promptingthat I could do.
I'm sure somebody listening tothis who's an AI expert would
like.
If only you had done thisprompt, it would have been
better.
But I was trying to put myselfinto like the average regular
person that was trying to writean SPCC for their facility and
(15:44):
then some of the storagecontainment volumes were just
mixed up again.
A good example of this is likewhen we prepare SPCC plans,
there's secondary containmentrequirements for all these
different kinds of containers.
Some of those requirements aregeneral, like to collect weeps,
drips and spills, you know, likethe spill bucket at the fill
point of your tank, for example.
(16:05):
But others are like sizedrequirements.
So you have a 330 gallonisotope.
You need to have containmentfor it that is equal to 330
gallons in size, right, plus 10%for precipitation if it's
outside and exposed tostormwater.
Well, depending on thecontainer type and how it's used
and where it's located, you cansort of have all these
(16:26):
different containmentapplications and it can be
really confusing.
So, for example, let's saywe've got a helicopter company
and they want to store 10, 55gallon drums in a remote
location as like a refuelingsupply station.
Right, if you're the helicoptercompany and you're building a
containment area for these 55gallon drums and there's 10 of
(16:47):
them does your secondarycontainment need to be 550
gallons, as big as everythingyou've got, 55 gallons for the
biggest container or just fivegallons to capture like weeps,
drips and leaks?
Cassie Kuzis (17:01):
I'm not sure off the top of my head.
Shannon Oelkers (17:04):
A lot of people , don't?
We get this question a lot.
Actually, under 40 CFR 112, thecorrect answer would be 55
gallons, the volume of thelargest container within the
containment area.
But if you don't have thatcontext or definition, using AI
to create your plan, we'llabsolutely spit out a wrong
answer.
In the prompt number two, wherewe had two tanks, it said that
(17:25):
the tanks had to havecontainment that was equal to,
you know, 4000 gallons plus 10%,right, 110%.
But it also said the tanks weredouble walled.
So if you were writing thisSPCC plan and getting ready to,
you know, buy a double walledtank, for example, you're
doubling up on the containmentand that's an expense you
wouldn't have needed to dobecause you've got double walled
(17:45):
tanks, which is your 110%containment already.
So that kind of complexitytheir AI is nowhere near
touching.
There's also a couple ofexamples where they listed the
broad line items for things like.
It would say things likeemployee training will be
provided for, spill response, orfigures showing the facility
layout will be included in anappendix, but that leaves out
(18:07):
any specifics for what thecontent of those things needs to
be and the SPCC is prettyspecific about.
You need to have the followingthings in your figures.
You need to have the followingpieces of your training program,
and so I feel like the spillresponse actions were somewhat
captured in the plans better,but spill prevention, like
transfer operations andmaintenance, and they were all
(18:29):
left out of all three of them.
They just didn't reference themat all.
They just didn't reference themat all.
I feel like, after going throughmultiple reviews with the EPA,
with our SPCC plans in general,these overgeneralized language
that AI is producing is just notgoing to be compliant in the
eyes of the EPA because itdoesn't address specific nuances
(18:50):
of each of the regulatoryrequirements.
And I guess a good example ofthis it's a difference between
check the truck for leaks andcheck the lowermost drains and
valves for leaks prior tofueling and prior to departure.
One of those is based on aregulatory language, the other
one is a overgeneralizedstatement and a lot of people
are like, oh yeah, I did, Ichecked the truck for leaks, but
(19:12):
the EPA, with 100% authority Ican say, will assign a violation
if you don't include thelowermost drains and valves in
your checklist and can show thatthat's part of your training to
the employee.
Cassie Kuzis (19:24):
Ah, so there's just a lot of nuance there, yeah
, and that's just like one tinypart Did any big surprises come
up?
Shannon Oelkers (19:32):
Yeah, there was a couple.
For whatever reason, AI gotportable drums and containers
pretty well.
I don't know if that's due tomore content about drum storage
and warehousing being availableon the internet or what, but it
got a lot of the portable drumsand container stuff pretty
accurate and pretty thoroughly.
Labeling and signage made anappearance on the third prompt.
(19:52):
It did not make an appearancein the first two, which both had
portable containers and tanks.
So I don't know why, but I didnotice that labeling and signage
finally made it on there on thethird time around.
It also got spill reportingreally well.
Again, I think likely becausethere's so much correct
information on the internetabout that specific task already
(20:12):
, because lots of people havehow to notify, how to do spill
notifications posted, you know,to the internet.
Cassie Kuzis (20:19):
Wow, that is really interesting, but it still
seems like for now I shouldprobably leave AI out of SPCCs.
Shannon Oelkers (20:26):
I really agree with that.
Honestly, I think if you wantedto create your own SPCC, just
reading the regulation line byline and responding to each
portion of that regulation wouldgive you much better results
than AI.
But in reality, I think mostcommercial facilities need a
professional plan writer, notjust reading it line by line,
because you need somebody whounderstands the regulations, the
(20:49):
context, your industry andoperations and that's really
going to turn into the bestoutcome for your facility
overall.
Cassie Kuzis (20:57):
Professionals, like I said, integrity.
Shannon Oelkers (21:00):
Of course you can't see me through the radio,
but I'm blushing modestly.
Yes, Professionals likeIntegrity or engineering groups,
you know you know youroperations best.
The regulations and theengineering standards and the
fire code all play into writingan SPCC plan.
I don't think it's typical formost people who are focused on
the main job task of storingfuel to also have a good grasp
(21:22):
of the fire code and engineeringstandards and inspection
standards, and so all of thatstuff feeds into.
If you have a professional thatcan help you and they're
working with you, you'll get thebest of both worlds.
You get your land-basedknowledge from being right there
on the ground doing the job,combined with their professional
knowledge about all of thesedifferent regulations and
standards.
Cassie Kuzis (21:41):
Well, do you think we should be worried about AI
in the future?
Will it ever be able to writethese plans?
Shannon Oelkers (21:47):
I think our jobs are safe from AI for now
and I have a feeling maybeforever, because facilities are
pretty straightforward when theyare built, but facilities
change as they age and I don'tknow that AI can overcome
facility specific content, likeit's got to overcome the as
(22:09):
designed right and then linkingthat at through time to the as
observed, and so the changes afacility goes through in 15
years, even if you fed the AI,your engineer design work work,
the tank farm is going to bedifferent and there's all these
like operational things abouthow they transfer fuel and the
ways that they do things andwhen and how they use drip pan.
(22:31):
You know, like there's just alot of things that I think that
human connector is always goingto need to do, because you know,
we know this from writing planswhat you write is never what
you find in reality, and so youneed to constantly connect that
you know, connect betweenreality and ideal, and I don't
think AI can do that without ahuman person.
Cassie Kuzis (22:50):
Well, that's a relief.
Shannon Oelkers (22:52):
Yes, that kind of wraps up what I found out
about AI.
We'll get those show notesposted.
Cassie Kuzis (23:01):
Yeah, thanks for doing that.
I'm sure a lot of people willbe interested in your results.
Shannon Oelkers (23:06):
Yeah, it should be fun to see what else people
see.
If you end up having commentsor you have some ideas about
this topic, we would love tohear them.
Go ahead and email us at infoat integrity dash envcom.
Or if you have ideas for futurepodcasts, I would really enjoy
hearing those.
We've had quite a few listenerideas in the past, so if you've
got one, let me have it.
Cassie Kuzis (23:27):
Awesome.
Well, thanks for that.
Shannon Oelkers (23:30):
Hi there, this is Shannon Oelkers and, as the
owner of Integrity Environmental, I wanted to take a minute here
at the end of the podcast tomake sure that you knew the
following this podcast is forinformational purposes only and
(23:54):
should not be considered legalor regulatory.
Thank you very much forlistening, and the views
expressed in this podcast maynot be those of the host that
would be me or IntegrityEnvironmental.
Thank you very much forlistening and if you do need
professional regulatory advice,we'd be happy to help you as
part of our consulting services.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.