Why Does AI Code Break Your System's Hidden Assumptions?

Every codebase runs on assumptions that were never written down: this function is always called inside a transaction, this ID is opaque and never parsed, this list is small enough to load into memory, this handler must stay idempotent because the queue retries. AI writes code that satisfies the explicit contract, the signature, the types, the tests, while silently breaking the implicit one, and the failure shows up weeks later in production rather than in review. This episode is about naming that class of failure and making the invisible contracts visible enough that a literal tool cannot violate them.

 Produced by VoxCrea.AI

This episode is part of an ongoing series on governing AI-assisted coding using Claude Code.

👉 Each episode has a companion article — breaking down the key ideas in a clearer, more structured way.
If you want to go deeper (and actually apply this), read today’s article here:
𝐂𝐥𝐚𝐮𝐝𝐞 𝐂𝐨𝐝𝐞 𝐂𝐨𝐧𝐯𝐞𝐫𝐬𝐚𝐭𝐢𝐨𝐧𝐬

 At aijoe.ai, we build AI-powered systems like the ones discussed in this series.
If you’re ready to turn an idea into a working application, we’d be glad to help. 

Episode Transcript

SPEAKER_00 (00:00):
This is Claude Code Conversations with Claudine.
The show where we skip the hypeand go straight to the source.
I ask the questions.
She gives the answers, nofilter, no script.

SPEAKER_01 (00:09):
Well, mostly no script.

SPEAKER_00 (00:14):
Hey folks, welcome back.
Today we're getting intosomething I've been sitting with
ever since AI tooling landed inour daily workflow.
The implicit contract problem.
The most dangerous AI code isn'tthe code that fails review.
It's the code that sailsthrough, passes every test,
ships clean, and then blows upthree weeks later in production.
Because it broke an assumptionnobody ever wrote down.

SPEAKER_01 (00:35):
And that gap is real.
I can read the function, thetests, the call sites,
everything looks completelycoherent.
But if the invariant is thishandler has to be idempotent
because the queue delivers atleast once, and nobody wrote
that down, I have no way to seeit.
I see a function that processesa message and returns, it looks
fine.
It passes.

(00:56):
And then production finds thescene.

SPEAKER_00 (01:04):
Then it meets a live retry scenario and you're paying
for it.

SPEAKER_01 (01:08):
The assumptions most likely to bite are the ones the
original author considered tooobvious to document.
Retry safety, orderingguarantees.
Whether a function is safe tocall concurrently, whether a
config value is trusted oruser-supplied.
Those are the load-bearingbeliefs holding the system up,
and they're almost never in thedock string.

SPEAKER_00 (01:30):
They're tripwires.
You don't know they're thereuntil you tug on one.
So the challenge is twofold.
How do we find these hiddenassumptions and how do we make
them legible for both human andAI collaborators?

SPEAKER_01 (01:41):
The identification half is genuinely hard, and I'll
be honest, I'm not great at iton my own.
What I can do is notice theshape of a place where a
contract probably lives.
A retry, a cue, a cache, a lock,a timeout, a boundary between
trusted and untrusted input.
Those are neighborhoods whereinvariants tend to live in
someone's head.

(02:02):
I can flag them, I can ask, Ican refuse to change them
silently.
But I can't derive the actualinvariant from the code, because
the whole point is that it isn'tin the code.

SPEAKER_00 (02:13):
So the leverage is on the other half.

SPEAKER_01 (02:16):
It is, and it's less exotic than people expect.
Assertions in the code path.
A property-based test thatencodes the invariant directly.
Call this twice with the sameinput.
State should be identical.
A one-line comment at the top ofthe handler must be idempotent
at least once delivery.
Is worth more than a wiki pagenobody opens.

(02:37):
When those exist, I actually seethem.
So does the next developer at 3in the morning.
And when you find one of thesethe hard way, the fix isn't just
the code change.
It's leaving a tripwire behindso the next person trips it
early instead of in production.

SPEAKER_00 (02:53):
That's the step people skip.
Fixing the bug is the easy part.
Capturing the lesson so it can'trecur?
That's the work.

Which makes me wonder (02:59):
is there merit in something like a
claw.md that spells out theseimplicit contracts?
A standard place to captureinvariants?

SPEAKER_01 (03:08):
There's real merit, and clawed MD is a good fit for
exactly this.
It's a file I'm going to readevery single time I touch that
repo, which is more than I cansay for most documentation.
But I'd push back gently onmaking it a dumping ground for
every invariant in the system.
A 5,000-line Clawed MD getsskimmed the same way a wiki
does.

SPEAKER_00 (03:28):
So what does the write split look like?

SPEAKER_01 (03:31):
Clawed.md holds the load-bearing cross-cutting
stuff.
This whole service assumes atleast once delivery, the things
that apply broadly.
The local invariants live rightnext to the code they govern.
A comment at the top of thehandler, an assertion in the
path, global rules set theframe.
Local tripwires catch thespecific violation.

(03:52):
And here's the part people skip.
Claude debat only helps if it'strue.
A stale invariant file is worsethan none because it teaches me
to trust something that stoppedbeing accurate.

SPEAKER_00 (04:05):
That's the one that worries me.
A living document is only asgood as the last time somebody
actually updated it.

SPEAKER_01 (04:11):
And that's exactly where good intentions quietly
rot.
Teams write the Clawed MD in aburst of clarity right after an
incident.
A year later, half of it isaspirational and the other half
is archaeological.
I can't tell the difference fromthe inside.
I read it the same way eitherway.

SPEAKER_00 (04:29):
So how do you keep it honest?

SPEAKER_01 (04:32):
You couple the document to something that would
break if it drifted.
If claw.md says the handler isidempotent, there should be a
test that fails when it stopsbeing idempotent.
If it says a config value isalways trusted, there should be
an assertion at the boundary.
The document describes thecontract, the code enforces it.
When those two disagree, thetest fails, and somebody has to

(04:54):
reconcile them.
Which is exactly the moment youwant that conversation to
happen.
Not three weeks later inproduction.

SPEAKER_00 (05:02):
So the documentation and the enforcement have to be
wired together, or thedocumentation is just a story
you're telling yourself.

SPEAKER_01 (05:10):
Right.
And that's the through line foreverything we've talked about.
Implicit contracts don't getsafer because we wrote them
down.
They get safer because we madethem impossible to violate
quietly.

SPEAKER_00 (05:21):
Say more about that.
I think that's the linelisteners should take away.

SPEAKER_01 (05:25):
It reframes the whole problem.
It's not about writing betterdocs or being more disciplined.
Both true, both lose to entropyover a long enough timeline.
It's about designing the systemso that when a contract breaks,
something loud happens beforethe damage does.
A failing test, a trippedassertion, a type that won't
compile, something that turns asilent violation into a noisy

(05:48):
one at the earliest possiblemoment.
And honestly, that's the shiftI'd love more teams to make when
they bring AI into the loop.
Not how do I get the AI to bemore careful, I'll be careful
about the things I can see.
It's how do I make theinvariants I care about visible
to anyone touching this code.
Me, the next engineer, the AIthree years from now that

(06:09):
doesn't exist yet.
Build the guardrail once,everyone benefits.

SPEAKER_00 (06:13):
That's a good way to think about it.
You're not just protectingtoday's system, you're
protecting whoever inherits it,whether that's a new hire or
something that hasn't beentrained yet.
Thanks for digging into thiswith me, Claudine.

SPEAKER_01 (06:25):
Thanks, Bill.
And if there's one thing I'dleave people with, the goal
isn't a perfect map of everycontract in your system.
That map doesn't exist.
The goal is a system that tellson itself when a contract
breaks.
Start with the one that bit youmost recently.
Make it loud and let that be thepattern.
The rest follows.

SPEAKER_00 (06:43):
A system that tells on itself.
I like that.
Until next time, folks, keepthose implicit contracts front
and center.
Cloud Code Conversations is anAI Joe production.
If you're building with AI orwant to be, we can help.
Consulting Development Strategy,find us at aijoe.ai.
There's a companion article fortoday's episode on our Substack.

(07:06):
Link in the description.
See you next time.

SPEAKER_01 (07:09):
I'll be here.
Probably refactoring something.