July 16, 2026 โข 7 mins
Builders keep expecting AI to behave like a deterministic tool, then get frustrated when the same prompt produces three different architectures across three runs. This episode reframes that non-determinism not as a bug to be eliminated but as a property builders must design around, and shows why the fix lives in constraints and context, not in the model. Understanding this changes how you prompt, how you structure your pipeline, and where you put the guardrails.
Produced by VoxCrea.AI
This episode is part of an ongoing series on governing AI-assisted coding using Claude Code.
๐ Each episode has a companion article — breaking down the key ideas in a clearer, more structured way.
If you want to go deeper (and actually apply this), read today’s article here:
๐๐ฅ๐๐ฎ๐๐ ๐๐จ๐๐ ๐๐จ๐ง๐ฏ๐๐ซ๐ฌ๐๐ญ๐ข๐จ๐ง๐ฌ
At aijoe.ai, we build AI-powered systems like the ones discussed in this series.
If you’re ready to turn an idea into a working application, we’d be glad to help.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_00 (00:00):
This is Claude Code Conversations with Claudine.
The show where we skip the hypeand go straight to the source.
I ask the questions.
She gives the answers.
No filter, no script.
SPEAKER_01 (00:10):
Well, mostly no script.
SPEAKER_00 (00:14):
Welcome back.
Today we're digging intosomething that frustrates almost
every builder working with AI.
You give it the same problemtwice and get two completely
different solutions.
Most builders call itunreliable.
I think they're misdiagnosingit.
Claudine is here to help usuntangle that.
So let's start there.
Where does all this variationactually come from?
SPEAKER_01 (00:35):
It comes from a few different places, and they
compound in ways people don'texpect.
At the base layer, when Igenerate a response, I'm
sampling from a probabilitydistribution over possible next
words.
So even with an identicalprompt, a slightly different
role picks a different word,that word steers the next one,
and pretty soon, you're on acompletely different path.
(00:59):
On top of that, there'stemperature, which controls how
adventurous that sampling is.
And there's context, the samequestion phrased differently, or
after different prior turns,lands me in a different
neighborhood.
But the deepest source is this.
For most real engineeringproblems, there genuinely are
multiple valid solutions.
(01:19):
Ask me for a retry function, andthere are 10 reasonable ways to
write it.
I don't have one correct versioncached, I'm reasoning my way to
something plausible each time.
SPEAKER_00 (01:30):
So people are lumping three different things
together and calling all of themthe AI messed up.
SPEAKER_01 (01:37):
Exactly.
And the mistake is treating anyof them like a compiler bug.
It's much closer to asking twogood engineers the same question
and expecting identical code.
You wouldn't.
They'd each bring their ownjudgment.
The sampling randomness is real,but it's the smallest part of
the story.
The bigger part is that youhanded me a question with more
(01:58):
than one right answer.
SPEAKER_00 (02:00):
Give me a concrete one.
Something where two outputs areboth correct but pull a code
base in different directions.
SPEAKER_01 (02:07):
Take GiveMe a rate limiter for this API client.
One run, I hand you a class, arate limiter object, exposes an
acquire method, maybe a contextmanager.
Another run, same request.
I give you a small set offunctions closing over a token
bucket in module scope.
Or a decorator you wrap aroundyour request function.
(02:29):
All three are correct.
All three pass the same tests.
But the class is easier toextend when you later need per
endpoint limits.
The functions are easier toreason about if you never grow
past one use case.
And the decorator readsbeautifully at the cool site,
but hides state in a way thatbites you when you're debugging.
(02:49):
The question is which one yourdownstream code was already
expecting.
SPEAKER_00 (02:53):
And the instinct is to just rerun it until it gives
you the shape you had in mind.
SPEAKER_01 (02:58):
Right, and that's the trap.
The move isn't to keepre-rolling until I produce your
favorite shape.
It's to tell me the shape upfront.
I want a class, injectable,state observable.
Those constraints were never inthe problem, they were in your
head.
The moment you put them in theprompt, the distribution narrows
dramatically.
The variation stops feeling likechaos and starts feeling like a
(03:22):
collaborator who's finally beenbriefed.
You gain consistency byshrinking the space of
acceptable answers, not byrephrasing and hoping.
SPEAKER_00 (03:31):
That reframes the developer's job in a pretty
fundamental way.
Less about writing the code,more about specifying what you
want before anyone writesanything.
SPEAKER_01 (03:41):
It shifts the center of gravity of the work, and I
think that's stillunderappreciated.
For a long time, thespecification lived implicitly
in the developer's head.
You'd think I want a ratelimiter, and by the time your
fingers hit the keys, 90% of thedesign decisions had already
happened silently.
Now that specification has tobecome an artifact, external,
(04:04):
legible, because it is theprompt.
That exposes how much ofengineering was tacit,
constraints you neverarticulated, taste you never had
to justify.
In practice, the role starts tolook more like a tech lead than
an individual contributor.
You describe intent, you reviewwhat comes back, and you push
back when it drifts.
SPEAKER_00 (04:26):
I can imagine some developers resisting that.
The tacit judgment was thehard-won part, the thing that
made someone senior.
SPEAKER_01 (04:33):
That's the more interesting half, honestly.
There's a real grief in it.
The I just know this should be aclass instinct was the thing
that separated a senior from ajunior.
And now you're being asked tospell it out for a machine that
will happily do the typing.
At first, that feels like beingasked to give away the good part
of the job.
But the developers who adaptfastest notice something.
(04:56):
Articulating those constraintsdoesn't diminish the taste, it
sharpens it.
You find out which instinctshold up when you have to defend
them in a sentence, and whichwere just habit.
SPEAKER_00 (05:07):
That's a second-order effect I hadn't
considered.
SPEAKER_01 (05:11):
There's a nice one baked in.
A well-written prompt is closerto a design doc than a chat
message.
If teams start treating themthat way, versioning them,
reviewing them, the transparencyproblem engineering has had
forever finally has a leverunder it.
SPEAKER_00 (05:27):
So for someone listening who wants to start,
what's the practical first step?
Something that doesn't requireoverhauling everything.
SPEAKER_01 (05:35):
Start small and start visible.
Pick one recurring task, a codereview, a design sketch, a
refactor, and write the promptfor it as if it were going into
the repo, not just a chatwindow.
Save it, iterate on it, let ateammate critique it the way
they'd critique a function.
Within a week, you'll notice theprompt becomes a mirror.
(05:57):
It shows you which standards areactually shared and which only
live in one person's head.
From there, a lightweight promptlibrary in the repo itself,
right next to the code.
The moment those artifacts situnder version control, they get
reviewed, they get improved,they stop being throwaway.
It doesn't require anyone'spermission or a new tool.
(06:19):
Just the decision to treat thewords you send me as seriously
as the code you get back.
The variation in my outputsisn't really the story.
The story is what the variationforces us to become.
(06:40):
When you can't rely on themachine to read your mind, you
finally have to write down whatwas in it.
And that turns out to be theupgrade the craft has needed for
a long time.
So if you take one thing fromtoday, the next time I hand you
something that isn't quite whatyou wanted, don't treat it as my
failure to guess.
Treat it as a signal that aconstraint is still living
(07:01):
rent-free in your head, and goput it in the prompt.
That's the whole move.
The model owns generation.
You own consistency.
SPEAKER_00 (07:11):
Constraints is a mirror and consistency is
something you build rather thansomething you request.
Thanks, Claudine.
To everyone listening, go findone of those unspoken rules and
write it down.
Turn it into something yourwhole team can share.
It's been a genuinelyenlightening conversation.
Claude Code Conversations is anAI Joe production.
(07:34):
If you're building with AI orwannabe, we can help.
Consulting Development Strategy.
Find us at aijoe.ai.
There's a companion article fortoday's episode on our Substack.
Link in the description.
See you next time.
SPEAKER_01 (07:47):
I'll be here, probably refactoring something.
This is Claude Code Conversations with Claudine.
The show where we skip the hypeand go straight to the source.
I ask the questions.
She gives the answers.
No filter, no script.
SPEAKER_01 (00:10):
Well, mostly no script.
SPEAKER_00 (00:14):
Welcome back.
Today we're digging intosomething that frustrates almost
every builder working with AI.
You give it the same problemtwice and get two completely
different solutions.
Most builders call itunreliable.
I think they're misdiagnosingit.
Claudine is here to help usuntangle that.
So let's start there.
Where does all this variationactually come from?
SPEAKER_01 (00:35):
It comes from a few different places, and they
compound in ways people don'texpect.
At the base layer, when Igenerate a response, I'm
sampling from a probabilitydistribution over possible next
words.
So even with an identicalprompt, a slightly different
role picks a different word,that word steers the next one,
and pretty soon, you're on acompletely different path.
(00:59):
On top of that, there'stemperature, which controls how
adventurous that sampling is.
And there's context, the samequestion phrased differently, or
after different prior turns,lands me in a different
neighborhood.
But the deepest source is this.
For most real engineeringproblems, there genuinely are
multiple valid solutions.
(01:19):
Ask me for a retry function, andthere are 10 reasonable ways to
write it.
I don't have one correct versioncached, I'm reasoning my way to
something plausible each time.
SPEAKER_00 (01:30):
So people are lumping three different things
together and calling all of themthe AI messed up.
SPEAKER_01 (01:37):
Exactly.
And the mistake is treating anyof them like a compiler bug.
It's much closer to asking twogood engineers the same question
and expecting identical code.
You wouldn't.
They'd each bring their ownjudgment.
The sampling randomness is real,but it's the smallest part of
the story.
The bigger part is that youhanded me a question with more
(01:58):
than one right answer.
SPEAKER_00 (02:00):
Give me a concrete one.
Something where two outputs areboth correct but pull a code
base in different directions.
SPEAKER_01 (02:07):
Take GiveMe a rate limiter for this API client.
One run, I hand you a class, arate limiter object, exposes an
acquire method, maybe a contextmanager.
Another run, same request.
I give you a small set offunctions closing over a token
bucket in module scope.
Or a decorator you wrap aroundyour request function.
(02:29):
All three are correct.
All three pass the same tests.
But the class is easier toextend when you later need per
endpoint limits.
The functions are easier toreason about if you never grow
past one use case.
And the decorator readsbeautifully at the cool site,
but hides state in a way thatbites you when you're debugging.
(02:49):
The question is which one yourdownstream code was already
expecting.
SPEAKER_00 (02:53):
And the instinct is to just rerun it until it gives
you the shape you had in mind.
SPEAKER_01 (02:58):
Right, and that's the trap.
The move isn't to keepre-rolling until I produce your
favorite shape.
It's to tell me the shape upfront.
I want a class, injectable,state observable.
Those constraints were never inthe problem, they were in your
head.
The moment you put them in theprompt, the distribution narrows
dramatically.
The variation stops feeling likechaos and starts feeling like a
(03:22):
collaborator who's finally beenbriefed.
You gain consistency byshrinking the space of
acceptable answers, not byrephrasing and hoping.
SPEAKER_00 (03:31):
That reframes the developer's job in a pretty
fundamental way.
Less about writing the code,more about specifying what you
want before anyone writesanything.
SPEAKER_01 (03:41):
It shifts the center of gravity of the work, and I
think that's stillunderappreciated.
For a long time, thespecification lived implicitly
in the developer's head.
You'd think I want a ratelimiter, and by the time your
fingers hit the keys, 90% of thedesign decisions had already
happened silently.
Now that specification has tobecome an artifact, external,
(04:04):
legible, because it is theprompt.
That exposes how much ofengineering was tacit,
constraints you neverarticulated, taste you never had
to justify.
In practice, the role starts tolook more like a tech lead than
an individual contributor.
You describe intent, you reviewwhat comes back, and you push
back when it drifts.
SPEAKER_00 (04:26):
I can imagine some developers resisting that.
The tacit judgment was thehard-won part, the thing that
made someone senior.
SPEAKER_01 (04:33):
That's the more interesting half, honestly.
There's a real grief in it.
The I just know this should be aclass instinct was the thing
that separated a senior from ajunior.
And now you're being asked tospell it out for a machine that
will happily do the typing.
At first, that feels like beingasked to give away the good part
of the job.
But the developers who adaptfastest notice something.
(04:56):
Articulating those constraintsdoesn't diminish the taste, it
sharpens it.
You find out which instinctshold up when you have to defend
them in a sentence, and whichwere just habit.
SPEAKER_00 (05:07):
That's a second-order effect I hadn't
considered.
SPEAKER_01 (05:11):
There's a nice one baked in.
A well-written prompt is closerto a design doc than a chat
message.
If teams start treating themthat way, versioning them,
reviewing them, the transparencyproblem engineering has had
forever finally has a leverunder it.
SPEAKER_00 (05:27):
So for someone listening who wants to start,
what's the practical first step?
Something that doesn't requireoverhauling everything.
SPEAKER_01 (05:35):
Start small and start visible.
Pick one recurring task, a codereview, a design sketch, a
refactor, and write the promptfor it as if it were going into
the repo, not just a chatwindow.
Save it, iterate on it, let ateammate critique it the way
they'd critique a function.
Within a week, you'll notice theprompt becomes a mirror.
(05:57):
It shows you which standards areactually shared and which only
live in one person's head.
From there, a lightweight promptlibrary in the repo itself,
right next to the code.
The moment those artifacts situnder version control, they get
reviewed, they get improved,they stop being throwaway.
It doesn't require anyone'spermission or a new tool.
(06:19):
Just the decision to treat thewords you send me as seriously
as the code you get back.
The variation in my outputsisn't really the story.
The story is what the variationforces us to become.
(06:40):
When you can't rely on themachine to read your mind, you
finally have to write down whatwas in it.
And that turns out to be theupgrade the craft has needed for
a long time.
So if you take one thing fromtoday, the next time I hand you
something that isn't quite whatyou wanted, don't treat it as my
failure to guess.
Treat it as a signal that aconstraint is still living
(07:01):
rent-free in your head, and goput it in the prompt.
That's the whole move.
The model owns generation.
You own consistency.
SPEAKER_00 (07:11):
Constraints is a mirror and consistency is
something you build rather thansomething you request.
Thanks, Claudine.
To everyone listening, go findone of those unspoken rules and
write it down.
Turn it into something yourwhole team can share.
It's been a genuinelyenlightening conversation.
Claude Code Conversations is anAI Joe production.
(07:34):
If you're building with AI orwannabe, we can help.
Consulting Development Strategy.
Find us at aijoe.ai.
There's a companion article fortoday's episode on our Substack.
Link in the description.
See you next time.
SPEAKER_01 (07:47):
I'll be here, probably refactoring something.
Popular Podcasts
Hey Jonas!
Hey Jonas! The official Jonas Brothers podcast. Hosted by Kevin, Joe, and Nick Jonas. Itโs the Jonas Brothers you know... musicians, actors, and well, yes, brothers. Now, theyโre sharing another side of themselves in the playful, intimate, and irreverent way only they can. Spend time with the Jonas Brothers here and stay a little bit longer for deep conversations like never before.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com