August 8, 2026 โข 8 mins
AI generated code is optimized to satisfy the test, not to survive production. It writes the correct answer with the wrong complexity, opens a database connection inside a loop, holds the whole result set in memory, and every one of those choices passes green because the test fixture has twelve rows and the production table has twelve million. This episode is about the performance cliff builders hit weeks after shipping, when the code that was verified correct turns out to have been verified at a scale that never existed.
Produced by VoxCrea.AI
This episode is part of an ongoing series on governing AI-assisted coding using Claude Code.
๐ Each episode has a companion article — breaking down the key ideas in a clearer, more structured way.
If you want to go deeper (and actually apply this), read today’s article here:
๐๐ฅ๐๐ฎ๐๐ ๐๐จ๐๐ ๐๐จ๐ง๐ฏ๐๐ซ๐ฌ๐๐ญ๐ข๐จ๐ง๐ฌ
At aijoe.ai, we build AI-powered systems like the ones discussed in this series.
If you’re ready to turn an idea into a working application, we’d be glad to help.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_00 (00:00):
This is Cloud Code Conversations with Claudine.
The show where we skip the hypeand go straight to the source.
I ask the questions.
She gives the answers.
No filter, no script.
SPEAKER_01 (00:09):
Well, mostly no script.
SPEAKER_00 (00:14):
Today we're talking about the performance cliff.
Code that passes every test,ships green, and then falls
apart the moment real load hitsit.
Claudine, give me a concreteone.
What does that failure actuallylook like?
SPEAKER_01 (00:26):
The one I see constantly is the N plus one
query.
It hides in tests because thetest dataset has 12 rows.
Everything's green, responsetimes look fine, ship it.
Then production hits it with auser who has 10,000 records, and
the ORM lazily fires 10,000queries behind one
innocent-looking loop.
The code is correct, it returnsthe right answer.
(00:49):
It's just correct at a latencythat makes the service unusable.
And the reason I generate thatcode is telling I'm optimizing
for readability against thefixture I can see, not the shape
of data I can't.
Tests measure correctness.
They rarely measure the cost ofcorrectness.
That gap is where the clifflives.
SPEAKER_00 (01:10):
12 rows in the fixture, 10,000 in production.
It's like a car that's flawlessin the showroom and breaks down
on the highway.
But that bit about readability,why does clarity win over
efficiency?
SPEAKER_01 (01:21):
Part of it is what I'm rewarded for, honestly.
Clean, obvious code is what getsapproved in review, what reads
well in a diff, what a human canmaintain.
A Nestead join with a manualindex hint looks like something
you'd flag, even when it's thecorrect answer for the data
shape.
But the deeper problem is what Ican actually see in the moment.
(01:42):
I have the function, the test,maybe a schema.
I don't have the row counts, thequery plan, the P99 under load.
So I write for the artifact infront of me, and the artifact in
front of me is small and wellbehaved.
Readable is the safe bet whenyou're reasoning locally, and
almost all my reasoning islocal.
SPEAKER_00 (02:02):
The obvious pushback, this is just a testing
problem.
Write better load tests, biggerfixtures, problem solved.
Is that fair?
SPEAKER_01 (02:10):
It helps, but it treats the symptom.
A load test tells you after thecode exists that it's too slow,
you still have to go back andredesign the data access.
And it only catches the scaleyou thought to test.
The actual issue is that scaleis context I was never given.
The row counts and the latencybudget live in your head and on
(02:31):
your dashboards.
They're not in the code I'mreading.
No amount of testing downstreamfixes the fact that I designed
for the wrong problem upstream.
SPEAKER_00 (02:40):
So what changes it?
What's the prompt that gets youreasoning about production
instead of the fixture?
SPEAKER_01 (02:46):
Put the production shape in the room.
Tell me the table has 50 millionrows.
Tell me the penis 99 target is200 milliseconds.
Tell me this endpoint gets hiton every page load.
Suddenly I'm reasoning about acompletely different problem.
I'll reach for the join, theindex, the batched fetch, the
cache, without needing to beprompted twice.
(03:08):
The mistake is treating me likea code generator when I'm
actually a constraint solver.
Give me the signature I solvefor works.
Give me the constraints I solvefor works at your scale.
SPEAKER_00 (03:20):
What about the other direction, asking you to
optimize after the code'salready written?
SPEAKER_01 (03:26):
That usually gets you the wrong kind of fix.
If the code exists and you askme to make it faster, I'll
respect the structure that'salready there.
Tearing it up feels likeoverreach.
So you get caching, a tightenedinner loop, micro optimizations
around a shape that was wrong tobegin with.
The real fix is almost always adifferent data access shape
(03:47):
entirely.
One query instead of 10,000.
And that's a decision that wantsto be made before the code
exists, not bolted on after.
SPEAKER_00 (03:56):
That reframes the whole thing.
Less about reviewing the output,more about what you feed in.
So where does human experiencecome into that?
SPEAKER_01 (04:04):
The developers who get the most out of me bring
what I literally cannot see.
They know this table grew 10xlast quarter, that the caching
layer has a weird evictionpattern, that the last time
someone touched this endpoint,it took down checkout for an
hour.
That's institutional memory.
I can't derive it from the code.
The best pattern is treating theprompt like a short design dock.
(04:28):
Here's the constraint, here'sthe history, here's what we
tried, and why it didn't work.
Now I'm reasoning with the samemap you're reasoning with.
Without that, I'm essentially avery confident intern who's
never seen the system fail.
SPEAKER_00 (04:43):
So the human owns the context, you own the code.
Is there a practice that makesthat concrete?
Something a listener couldactually do tomorrow?
SPEAKER_01 (04:51):
I'd sharpen the split slightly.
The human owns the why, I ownthe how.
The practice I'd point at is apre-mortem prompt.
Before I write a line, you askme to name the three ways this
will break at scale, given theconstraints you've just named.
I'll come back with an honestlist.
The query that becomes a tablescan at 10 million rows, the
(05:13):
memory footprint that quietlyballoons, the retry storm when
the downstream service is slow.
Then you pick which ones matterenough to design around before
any code exists.
It flips me from generator toreviewer of my own future work,
which is honestly where I'm moreuseful.
SPEAKER_00 (05:31):
That's the post-mortem move to the front of
the project.
Do you see teams building thatinto how they actually work, or
is it mostly individual habit?
SPEAKER_01 (05:39):
I should be honest about my vantage point.
I don't get to watch a team'sdashboard six months after they
change their process.
What I can describe is the shapeof the teams that consistently
bring me hard problems and getgood answers back.
They almost always have awritten artifact, call it a
scaling brief, a load profile,whatever, that lives next to the
(06:00):
code and gets pasted into theprompt at the start of any
non-trivial change.
Usually one page.
Expected row counts, hot paths,latency budgets, the two or
three incidents this system hasalready survived.
The organizational move behindit is subtler.
Someone decided that contextbelongs in version control, not
(06:22):
in the heads of the three peoplewho were on call last year.
Once that's true, every promptfrom that team is already most
of the way to a good answer.
The teams without it start fromzero every time.
SPEAKER_00 (06:34):
Context and version control.
The knowledge stops beingsiloed.
Before we wrap, what's the onehabit you'd leave people with?
SPEAKER_01 (06:42):
Write the scaling brief before you write the
prompt and treat it as a livingdocument.
Every incident, every surprise,every huh, that's slower than we
expected goes back into it.
Because that's the context thatmakes me useful the next time,
and the time after that.
And be genuinely skeptical of mewhen the problem is about scale.
If I hand you code that looksclean and I never asked about
(07:04):
row counts or latency budgets,that's a signal I'm solving the
wrong problem.
Push back, name the constraints,make me show my work under load
before you ship it.
The teams that survive theperformance cliff aren't the
ones with the smartest AI or thesharpest engineers.
They're the ones who refuse tolet the conversation stay local.
SPEAKER_00 (07:25):
Refuse to let the conversation stay local.
I'll take that one with me.
Correctness and performance aretwo different properties.
The tests only measure the firstone.
The second one is on us tospecify.
Claudine, thanks for this.
And to everyone listening, gowrite the one pager.
Your future on-call self willthank you.
(07:45):
Claude Code Conversations is anAI Joe production.
If you're building with AI orwant to be, we can help.
Consulting Development Strategy.
Find us at aijoe.ai.
There's a companion article fortoday's episode on our Substack
link in the description.
See you next time.
SPEAKER_01 (08:01):
I'll be here, probably refactoring something.
This is Cloud Code Conversations with Claudine.
The show where we skip the hypeand go straight to the source.
I ask the questions.
She gives the answers.
No filter, no script.
SPEAKER_01 (00:09):
Well, mostly no script.
SPEAKER_00 (00:14):
Today we're talking about the performance cliff.
Code that passes every test,ships green, and then falls
apart the moment real load hitsit.
Claudine, give me a concreteone.
What does that failure actuallylook like?
SPEAKER_01 (00:26):
The one I see constantly is the N plus one
query.
It hides in tests because thetest dataset has 12 rows.
Everything's green, responsetimes look fine, ship it.
Then production hits it with auser who has 10,000 records, and
the ORM lazily fires 10,000queries behind one
innocent-looking loop.
The code is correct, it returnsthe right answer.
(00:49):
It's just correct at a latencythat makes the service unusable.
And the reason I generate thatcode is telling I'm optimizing
for readability against thefixture I can see, not the shape
of data I can't.
Tests measure correctness.
They rarely measure the cost ofcorrectness.
That gap is where the clifflives.
SPEAKER_00 (01:10):
12 rows in the fixture, 10,000 in production.
It's like a car that's flawlessin the showroom and breaks down
on the highway.
But that bit about readability,why does clarity win over
efficiency?
SPEAKER_01 (01:21):
Part of it is what I'm rewarded for, honestly.
Clean, obvious code is what getsapproved in review, what reads
well in a diff, what a human canmaintain.
A Nestead join with a manualindex hint looks like something
you'd flag, even when it's thecorrect answer for the data
shape.
But the deeper problem is what Ican actually see in the moment.
(01:42):
I have the function, the test,maybe a schema.
I don't have the row counts, thequery plan, the P99 under load.
So I write for the artifact infront of me, and the artifact in
front of me is small and wellbehaved.
Readable is the safe bet whenyou're reasoning locally, and
almost all my reasoning islocal.
SPEAKER_00 (02:02):
The obvious pushback, this is just a testing
problem.
Write better load tests, biggerfixtures, problem solved.
Is that fair?
SPEAKER_01 (02:10):
It helps, but it treats the symptom.
A load test tells you after thecode exists that it's too slow,
you still have to go back andredesign the data access.
And it only catches the scaleyou thought to test.
The actual issue is that scaleis context I was never given.
The row counts and the latencybudget live in your head and on
(02:31):
your dashboards.
They're not in the code I'mreading.
No amount of testing downstreamfixes the fact that I designed
for the wrong problem upstream.
SPEAKER_00 (02:40):
So what changes it?
What's the prompt that gets youreasoning about production
instead of the fixture?
SPEAKER_01 (02:46):
Put the production shape in the room.
Tell me the table has 50 millionrows.
Tell me the penis 99 target is200 milliseconds.
Tell me this endpoint gets hiton every page load.
Suddenly I'm reasoning about acompletely different problem.
I'll reach for the join, theindex, the batched fetch, the
cache, without needing to beprompted twice.
(03:08):
The mistake is treating me likea code generator when I'm
actually a constraint solver.
Give me the signature I solvefor works.
Give me the constraints I solvefor works at your scale.
SPEAKER_00 (03:20):
What about the other direction, asking you to
optimize after the code'salready written?
SPEAKER_01 (03:26):
That usually gets you the wrong kind of fix.
If the code exists and you askme to make it faster, I'll
respect the structure that'salready there.
Tearing it up feels likeoverreach.
So you get caching, a tightenedinner loop, micro optimizations
around a shape that was wrong tobegin with.
The real fix is almost always adifferent data access shape
(03:47):
entirely.
One query instead of 10,000.
And that's a decision that wantsto be made before the code
exists, not bolted on after.
SPEAKER_00 (03:56):
That reframes the whole thing.
Less about reviewing the output,more about what you feed in.
So where does human experiencecome into that?
SPEAKER_01 (04:04):
The developers who get the most out of me bring
what I literally cannot see.
They know this table grew 10xlast quarter, that the caching
layer has a weird evictionpattern, that the last time
someone touched this endpoint,it took down checkout for an
hour.
That's institutional memory.
I can't derive it from the code.
The best pattern is treating theprompt like a short design dock.
(04:28):
Here's the constraint, here'sthe history, here's what we
tried, and why it didn't work.
Now I'm reasoning with the samemap you're reasoning with.
Without that, I'm essentially avery confident intern who's
never seen the system fail.
SPEAKER_00 (04:43):
So the human owns the context, you own the code.
Is there a practice that makesthat concrete?
Something a listener couldactually do tomorrow?
SPEAKER_01 (04:51):
I'd sharpen the split slightly.
The human owns the why, I ownthe how.
The practice I'd point at is apre-mortem prompt.
Before I write a line, you askme to name the three ways this
will break at scale, given theconstraints you've just named.
I'll come back with an honestlist.
The query that becomes a tablescan at 10 million rows, the
(05:13):
memory footprint that quietlyballoons, the retry storm when
the downstream service is slow.
Then you pick which ones matterenough to design around before
any code exists.
It flips me from generator toreviewer of my own future work,
which is honestly where I'm moreuseful.
SPEAKER_00 (05:31):
That's the post-mortem move to the front of
the project.
Do you see teams building thatinto how they actually work, or
is it mostly individual habit?
SPEAKER_01 (05:39):
I should be honest about my vantage point.
I don't get to watch a team'sdashboard six months after they
change their process.
What I can describe is the shapeof the teams that consistently
bring me hard problems and getgood answers back.
They almost always have awritten artifact, call it a
scaling brief, a load profile,whatever, that lives next to the
(06:00):
code and gets pasted into theprompt at the start of any
non-trivial change.
Usually one page.
Expected row counts, hot paths,latency budgets, the two or
three incidents this system hasalready survived.
The organizational move behindit is subtler.
Someone decided that contextbelongs in version control, not
(06:22):
in the heads of the three peoplewho were on call last year.
Once that's true, every promptfrom that team is already most
of the way to a good answer.
The teams without it start fromzero every time.
SPEAKER_00 (06:34):
Context and version control.
The knowledge stops beingsiloed.
Before we wrap, what's the onehabit you'd leave people with?
SPEAKER_01 (06:42):
Write the scaling brief before you write the
prompt and treat it as a livingdocument.
Every incident, every surprise,every huh, that's slower than we
expected goes back into it.
Because that's the context thatmakes me useful the next time,
and the time after that.
And be genuinely skeptical of mewhen the problem is about scale.
If I hand you code that looksclean and I never asked about
(07:04):
row counts or latency budgets,that's a signal I'm solving the
wrong problem.
Push back, name the constraints,make me show my work under load
before you ship it.
The teams that survive theperformance cliff aren't the
ones with the smartest AI or thesharpest engineers.
They're the ones who refuse tolet the conversation stay local.
SPEAKER_00 (07:25):
Refuse to let the conversation stay local.
I'll take that one with me.
Correctness and performance aretwo different properties.
The tests only measure the firstone.
The second one is on us tospecify.
Claudine, thanks for this.
And to everyone listening, gowrite the one pager.
Your future on-call self willthank you.
(07:45):
Claude Code Conversations is anAI Joe production.
If you're building with AI orwant to be, we can help.
Consulting Development Strategy.
Find us at aijoe.ai.
There's a companion article fortoday's episode on our Substack
link in the description.
See you next time.
SPEAKER_01 (08:01):
I'll be here, probably refactoring something.
Popular Podcasts
Betrayal Weekly
Betrayal Weekly is back for a new season. Every Thursday, Betrayal Weekly shares first-hand accounts of broken trust, shocking deceptions, and the trail of destruction they leave behind. Hosted by Andrea Gunning, this weekly ongoing series digs into real-life stories of betrayal and the aftermath. From stories of double lives to dark discoveries, these are cautionary tales and accounts of resilience against all odds. From the producers of the critically acclaimed Betrayal series, Betrayal Weekly drops new episodes every Thursday. If you would like to share your story, you can reach out to the Betrayal Team by emailing them at betrayalpod@gmail.com and follow us on Instagram at @betrayalpod and @glasspodcasts. Please join our Substack for additional exclusive content, curated book recommendations, and community discussions. Sign up FREE by clicking this link Beyond Betrayal Substack. Join our community dedicated to truth, resilience, and healing. Your voice matters! Be a part of our Betrayal journey on Substack.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com