June 10, 2026 • 19 mins
In this episode, the team gets hands-on with vibe coding and what it actually looks like inside real data work. From building API connections to generating synthetic datasets, they walk through where AI is genuinely speeding things up and where it still runs into friction.
The big takeaway: writing code might be faster, but everything around the data still matters just as much.
Blue Margin helps growing companies make better decisions with their data. From building data pipelines to creating reporting and dashboards, we handle the technical side so teams can focus on using their data—not chasing it.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:05):
Welcome to the Dashboard EffectPodcast.
I'm Brick Thompson.
And I'm Landon Oakes.
Hey, Landon.
So we thought today we'd talk abit about vibe coding using AI.
I've been doing some personallyover the last few months,
especially since about I don'tknow, Thanksgiving, Christmas
time last year when I think itwas Opus 4.5 came out.
(00:27):
It was pretty darn good.
All of a sudden Claude Code wasreally good.
I don't do any coding reallyprofessionally at all.
Haven't for you know a coupledecades at least.
Not probably a couple decades.
But all of a sudden, I'm doingall kinds of coding.
And it's really fun to be ableto do that.
And then when Opus 4.6 came outfrom Anthropic, I think it was
(00:48):
February 6th.
Uh same time GPT 5.4 came out.
Anyway, I was I was still doingthe vibe coding there using uh
cloud code, still fantastic.
Now, just yesterday we got GPT5.5 out.
Codex, you know, the the OpenAIapp is updated.
I'll probably be spending myweekend doing that.
So anyway, I thought we werejust talking about it.
(01:08):
We're doing uh not really vibecoding here at Blue Margin, but
we are using AI tools to help usbuild our stuff.
I know you've been experimentingquite a lot.
Yeah.
Um maybe maybe we can startthere.
We just talk about what we'redoing, what we're finding is
helpful, where it's workingwell, where we still have to
exercise a lot of care.
Yeah, definitely.
Let's see.
(01:29):
So and we've we've done severalthings recently from all the way
from uh you know automating, notnecessarily automating, but
vibecoding a uh API notebookthat that pulls data out of out
of a source, um, which has beeninteresting.
We've you know created syntheticdata sets and demos, demo data
sets, because one of thestruggles is you want a data set
(01:52):
that replicates a business, butit you also want one that's
updating and new.
Yeah.
Um so you can kind of createyour own now without a ton of
effort, which has been reallycool.
Um Yeah, several more that I'mprobably not thinking of.
It is cool.
So you you manage our platformteam, and so you're you and your
guys manage all of our clients'data lake houses, but a big part
(02:14):
of that is pulling in new datasources.
So creating uh you know a reallyrobust data pipeline that can
run daily or multiple times aday and has a graceful fail mode
if something goes wrong so itdoesn't leave the data lake in
bad shape and sets off alertsand you know makes it easy for
the platform engineers, well,easier, to go in and fix it.
(02:37):
But also right up front, when anew data source comes from one
of our clients, we've got tomake that connection.
So we've got a series of Jupyternotebooks, depending on what
type of data source it is, so wecan make that connection.
In some cases, um, I think Idon't know, somewhere like a
hundred data sources.
We have something we've usedbefore, so we at least something
have something we can startwith.
(02:57):
Um sometimes we get brand newstuff though, like uh like a
totally novel API.
It's not uncommon for APIdocumentation to not be great.
Yeah.
And so, yeah, that's a perfectplace to use vibe coding.
Um, but we want to start withour standards.
We have best practices, we wantto do all those things I was
talking about.
So how do you how do you start?
(03:18):
How do you get uh your I don't Ithink you guys are using VS Code
with Claude.
Yeah, if I'm if I recall.
So how do you get it to knowwhat your target architecture is
and make sure it does a good joband doesn't go off and do
something kind of crazy?
Yeah, yeah.
So it's still a work inprogress.
We're still refining it, whichI'm sure all things AI are going
to be that way.
(03:39):
But um right now we're usingskills.
So we've you know kind oftrained skills to uh learn what
our current best practicetemplate is.
So what some of our best Pythonengineers made.
Um we gave it that and we said,hey, this is the stuff you need
to follow.
Here are the things youshouldn't do, here are the
things you should do.
Um it's actually quiteinteresting because we we ended
(04:01):
up trying to put everything inone skill.
And that so that includes likehow do I handle all these edge
cases?
How do I go out and finddocumentation?
What do I do if I can't find thedocumentation?
Yeah.
Okay.
Yeah.
And we found some weird behaviorwith that.
It's almost like we just gave ittoo much.
Yeah.
So one of the things we'reexperimenting with now, which
(04:24):
seems a little bit morepromising, is having discrete
skills that can call each other.
Oh, that's okay.
Do you have like an orchestratorskill that knows which skills to
call?
So, like, for example, I'mimagining you can tell me if
this is what you're doing ornot, but um you have a new API,
you can have a skill that goesout and reads the API
documentation.
Exactly.
For example, that's actuallywhat what led us to the idea
(04:47):
because we are having a lot oflot of trouble with the API
documentation, because not alldocumentation is AI friendly.
I just can't read it becausethey put it in JavaScript or
there's some weird backend onit.
So Yeah, interesting.
And some of these APIs have sortof secret off-the-menu stuff
that you have to know to reallymake it work well.
Yeah.
(05:07):
Yeah.
Yeah.
And then sometimes the APIsdon't work well, so you end up
going to the old ODBC orsomething.
Yeah.
Anyway, yeah.
So um I know you've got goalsaround bringing down the amount
of time it'll take to create oneof those.
One of the interesting things uhto me that uh you told me you
found out was that it's notnecessarily the hands-on
keyboard time, but there's a lotof time around that, getting
(05:29):
access to the data source,getting permissions set up
right, getting the OAuth set up,you know, uh the system
accounts, whatever you need, umuh as well as you know, looking
at code and and doing reviewsand all that.
So the hands-on keyboard timecan come down significantly, but
you can still end up spendingdays.
Like may maybe hands-on keyboardis half a day now.
(05:50):
Yeah.
But you can still spend days oreven a couple weeks getting it
done because of all thatancillary stuff.
Yeah.
And I sort of wonder if that'sgoing to be the case with a lot
of business agentics stuff for awhile.
Yeah.
I think it probably will,actually.
Yeah, I would I wouldn't besurprised.
You know, it's whenever you needto coordinate people, really.
That's where a lot of time a lotof times our roadblocks come
(06:12):
from.
So Yeah.
Yeah, it's true.
It's gonna it's gonna beinteresting to see how all of
that plays out.
Like if you tried to build askill that said, okay, contact
the client, get them to set upthe author, it's just not gonna
work because you need to talkthrough that very uh variable
thing that happens, I think.
Yeah, yeah.
And it's you know, we we've seenjust hundreds and hundreds and
(06:35):
hundreds of sources, and it'sjust always so completely
different.
Yeah.
You know, sometimes there'snothing even online about it,
right?
Yeah.
We've run into that before.
People just don't talk about itonline.
So the AI doesn't know, doesn'tisn't trained on it, it doesn't
know where to go find anythingabout it.
Kind of runs into a dead end.
Yeah.
Do you think does Claude Code doa pretty good job of following
(06:56):
our best practices?
It's I don't know.
I'd say like 80-20.
I might my team might say I'mbeing too generous to Claude.
Really?
Yeah.
Um they uh they have some oftheir times where it's like, oh
well, it just went over here andedited that thing that I didn't
want it to edit, right?
Right.
And so that's part of one of thereasons why we actually switched
(07:18):
to VS Code.
Um from just using the terminal.
Yeah, exactly.
Because one nice thing is thatas long as you don't tell it,
yeah, just go crazy, editwhatever the heck you want.
Um, you say, Let me approveedits, it will show you a
side-by-side comparison.
Here's your version, here's whatI'm gonna change, and it
highlights the changes for you.
You know, like those of you whomight have used um Git compare.
(07:41):
It's that essentially you'recomparing what the AI did to
what you have.
Yeah.
And so makes it way easier tomake sure it doesn't go off the
rails and edit somethingcompletely different, along with
sneaking in, you know, theediting you asked for.
I hate that.
Oh, and that was the problem,you know, six months, nine
months ago, whenever I wouldplay with vibe coding, because I
have been for, I don't know, thelast year or so.
(08:02):
But not that consistently, maybefor the first half of last year,
till we got up to the holidayseason, because it just wasn't
that good.
Yeah.
You know, it would producesomething that was close, and
then you'd risk r request fixes,and it would maybe make the fix
but break something else.
And it was just sort of it waslike playing whack-a-mole.
And then all of a sudden, aroundThanksgiving, Christmas time, it
(08:23):
stopped feeling like that.
And then around February, itreally stopped feeling like
that.
Yeah.
Yeah.
It's gotten so much better.
Yeah.
So uh well, you're gonna have totry GPT 5.5, the codex.
Yeah.
Uh and uh and let me know howthat is.
I'll be playing with that too.
Another thing I know you wereworking on, so we've got I'm
doing work for clients toconnect Claude and other um
(08:47):
agents to a data lakehouse, totheir data lake house, and be
able to do ad hoc queries forexecutives to be able to do, you
know, get quick answers onfinancial data or operational
data.
And it seems like that would beeasy, but um it's easy to get
wrong answers, actually.
And so we've been buildingsomething called a platinum
layer, which is to take the goldlayer out of the medallion
(09:09):
architecture and then uh do somethings to it, denormalize it
even more, rename columns,rename tables, make sure all of
our joins are done in aconsistent manner, and then
actually stack a bunch ofmarkdown files into the data
lake as well with context.
But then in addition to that,um, we started building custom
MCP servers for differentclients that that have a lot
(09:31):
more instructions about how togo and get an answer for the
agent.
Um, it's been reallyinteresting, and you've been
doing some work to build a demofor that.
Yeah.
And part of that was sort ofvibe coding a data set, right?
Because we don't want to use ourdata or any of our clients'
data, obviously, um to demo.
So we had to come up withcompletely synthetic data.
(09:52):
And there's things like umAdventureWorks out there or
Contoso or whatever, you know,the Microsoft thing.
So there's there's a lot ofstuff.
But but you actually startedfrom scratch.
Why don't you talk about that alittle bit?
I thought it was pretty cool.
Yeah, definitely.
So as the as the idea to startfrom scratch actually came from
AI itself.
Um, I I I just started asking itlike, how do we get around this
(10:13):
problem?
Help me brainstorm this, please.
Got that idea from it, and thenwas like, yes, let's go for it.
Um so I basically vibe codedit's it's nothing crazy, but
it's a it's a SQL script thatbuilds out tables, generates
random data, generates customersales reps, products, et cetera,
targets, um which then you justkind of run it.
(10:34):
It takes like 20 minutes becauseyou know, it it worked worked at
first.
And then where I got myself introuble is I started trying to
tweak it to try to make it seemmore real.
It's really hard to make likeseasonality.
Well, seasonality actually iseasy.
That's that's not hard.
Okay.
But like a customer leaving andthen maybe returning a year
(10:56):
later, right?
Stories like that, really,really hard to find.
Yeah.
Um, so I uh was trying to promptit to do that, and I ended up
destroying pretty much justwiping out the thing.
I mean, I didn't wreck itcompletely, but it made zero
sense.
Like I'd go ask, because I useClaude to connect to our MCP
server to talk to the data, andI'd be like, hey, tell me our
you know sales on this customerversus their target.
(11:19):
And it's like, well, they are2,000% above their target.
You should talk to the personwho set these targets.
I don't know what they werethinking.
Like, oh, oops.
So that was when I learned thatI really want Git to back up my
code base when I'm Oh, youweren't you didn't have that in
a repository?
I had to start from scratch.
Oh, that sucks.
(11:40):
Yeah, it was painful.
And we use we use uh AzureDevOps, right?
Yeah.
Are you s are you starting touse Git now that you're doing
more Vibe code?
I still use DevOps.
I mean it's Git on the end.
Um Yeah, I would like to getmore into Git, but we have we
have DevOps licenses.
It's kind of it's free for us,right?
So it is Git on DevOps.
So I just I guess I mean GitHub.
(12:01):
Yeah.
Yeah.
Yeah.
That's all my hobby stuff I'vebeen doing on that.
Yeah.
All right.
So then how did you get it?
So you started over, I guess.
Started from scratch,essentially.
Okay.
Pre-prompted, just just wipewiped it all out, started from
scratch, connected it to a repo,um, so I can commit changes and
roll back.
Yeah.
But yeah, it was a good learningexperience for sure.
(12:22):
Aaron Powell So one of theproblems with sample data sets
is that they're static usually,almost always.
And so as you're using them totest and so on, it's for data
that's six months old or twoyears old.
So have you figured out a way tosort of keep adding fresh
records?
Yeah, yeah, definitely.
So that was one of the keyrequirements I gave it.
You know, I that's one thingtoo.
(12:44):
Like when you Vive Code, yougotta start with really clear.
This is what it needs, this iswhat the end product needs to
look like.
Here are the things that itneeds to do, et cetera.
The clearer you can be, thebetter.
It created a pretty intense, Imean it's not intense in terms
of like sloppy, it's actuallyreally good uh procedure that
you call and it goes throughseveral different conditions.
(13:05):
It has seasonality built in andhas customer targets and
segments and kind of weightswhere their purchases go
depending on attributes on thecustomer.
Um and you can tell it, I wantto generate a thousand orders a
day, I want to generate 10orders a day, et cetera.
That's cool.
And so it was it was pretty coolto see because you know, with
Python, I immediately attachedto this is so easy to, you know,
(13:28):
like coding typical coding.
I know SQL's coding, but kind ofin a different bit.
Um immediately attached to howyou can do that.
With SQL, it's always been alittle harder to figure out how
do I get this thing to writegood SQL because you need that
data context of it.
Um data and you almost you haveto have a business context too,
right?
(13:48):
Exactly.
So you know, here's here's theconditions I'm trying to
simulate so that I can have aninteresting demo.
Yeah, exactly.
But basically it it's uh youknow, I consider myself a really
strong SQL developer.
I've been doing it for years andyears.
You are truly an expert, yeah.
And yeah, the stuff in there waswas insane, you know.
(14:08):
It was stuff that I knew, butthe way it put it all together
and in creative ways was wasreally cool to see.
Oh, that's cool.
How much debugging did you haveto do?
On that proc?
Yeah.
Yeah.
I went back and forth withClaude.
I don't have a number, but likeprobably three hours or so back
and forth.
But the thing is, you know, I'dhave to go go in there, then I'd
(14:29):
have to run everything, um, waitfor it to populate, and then
check the results.
Sometimes I'd export CSVs andsend it back to Claude.
I'd ask it like, hey, write me aquery that will summarize this
so we can make sure this makessense.
And I just paste the query in,you know, and run it.
Did you have it create a test uhtest suite to run?
I did not for that onespecifically, mainly because I
(14:52):
need to still figure out how toconnect AI to Well, I guess we
do know how to do it now.
To the data later.
Which right now, yes, I can dothat.
So that's a great idea.
Next time I'm gonna have it justrun its own tests, get its own
data back, adjust it.
Yeah.
It's a great idea.
So what's next for you with theteam here in terms of uh AI
(15:15):
coding?
Yeah, so you know, we're justtrying to get as efficient as we
possibly can with uh ourday-to-day work, right?
So a couple of things, you know,we're redesigning our standard
templates, um, we're making it abit more modular so that AI has
less to edit.
Right.
Okay.
Um there's a couple of drivingforces on that.
(15:36):
You know, obviously the less weneed to edit, the less context
it has to use on it.
Yeah.
Um more targeted it can be.
And then also, you know, ifprices increase like crazy for
some reason, we're not we're notcompletely underwater.
Right.
Um and then the other one that Ireally want to do too that I've
heard success about is umgetting just automated bug
(15:58):
fixes, right?
So pipeline for a client failsovernight, it's going to
automatically go look at thecode in our repos, um, make what
it thinks is the fix, put a pushuh pull request in, and then in
the morning we just look at it,right?
We'll still want that human inthe loop for for now.
Um that's pretty cool.
But I know some you can do acode review too with a
(16:20):
recommended, you know, I've I'vereviewed the pull request.
Yeah.
I recommend that this one beapproved.
Here's why.
Yeah you still might you stillmight look at it, but it might
speed you up.
Yeah, if we got trusty enoughwith it, yeah, maybe we have a
reviewer that reviews all therequests.
Yeah.
If it's ends up being goodenough, it can just
automatically push it and rerunit.
I'd be crazy.
I think these tools are gettingreally good on coding where you
(16:42):
can almost get there.
I mean you have to obviously besuper careful around anything
that has to do with security,exactly.
Or that's super important umoperational data.
You don't want to put bad datain the data lake in the lake
house.
Um but because it's funny, I wasreading an article about why do
LLMs hallucinate so much just infree text conversations or
(17:03):
summarizing dictation orwhatever, but in code it really
nails it.
And and some of the theorieswere that, well, there's so much
code out there that you cantrain way, way more than sort of
random um prose that's outthere.
And it's way more structured.
And the languages are veryconfined, like the the total
(17:24):
vocabulary and the syntax is notthat big.
So they can get really good atit.
And so it's not like the problemthey're seeing in law where you
know 75% of big legal briefswritten with uh with AI have
hallucinated stuff in them,made-up cases.
And I mean it just happenedagain to another big law firm,
amazingly.
(17:45):
Um it's it's different thanthat.
Not that it can't do thingswrong or start going down a bad
path or introduce some some bug,but it's amazing how good it's
gotten this last, I don't know,six months.
Yeah.
Just as really complete stepchange.
Oh, yeah, I completely agree.
I mean, it's got public reposout there all over the place.
Stack Overflow is a gold mine ofinfo now.
(18:07):
Yeah.
Yeah.
I've heard they're actually umselling their data now as
training data.
Oh, that's why models are that'swhere you used to go.
If you had a question, that'swhere you went.
Yeah, exactly.
I used to go there every singleday.
I'd always have Stack Overflowopen.
I haven't opened it in four orfive, probably more than that,
probably like six months at thispoint.
Yeah, I'm not surprised.
(18:28):
I just get answers faster, youknow, less digging.
I think next time I'd love totalk about sort of culturally
what it's been like to get theteam on board with that because
there's a there's a wide rangeof reactions to from oh, I can't
wait to do all write all my codethat way to gosh, one of the
things I really enjoy about myjob is crafting code.
Like it's almost an art.
(18:48):
So I don't really want to givethat up.
So anyway, we'll talk about thatnext time.
Yeah, absolutely.
But uh good discussion and uhtalk to you soon.
Yeah, thank you.
Appreciate it.
Talk to you later.
Welcome to the Dashboard EffectPodcast.
I'm Brick Thompson.
And I'm Landon Oakes.
Hey, Landon.
So we thought today we'd talk abit about vibe coding using AI.
I've been doing some personallyover the last few months,
especially since about I don'tknow, Thanksgiving, Christmas
time last year when I think itwas Opus 4.5 came out.
(00:27):
It was pretty darn good.
All of a sudden Claude Code wasreally good.
I don't do any coding reallyprofessionally at all.
Haven't for you know a coupledecades at least.
Not probably a couple decades.
But all of a sudden, I'm doingall kinds of coding.
And it's really fun to be ableto do that.
And then when Opus 4.6 came outfrom Anthropic, I think it was
(00:48):
February 6th.
Uh same time GPT 5.4 came out.
Anyway, I was I was still doingthe vibe coding there using uh
cloud code, still fantastic.
Now, just yesterday we got GPT5.5 out.
Codex, you know, the the OpenAIapp is updated.
I'll probably be spending myweekend doing that.
So anyway, I thought we werejust talking about it.
(01:08):
We're doing uh not really vibecoding here at Blue Margin, but
we are using AI tools to help usbuild our stuff.
I know you've been experimentingquite a lot.
Yeah.
Um maybe maybe we can startthere.
We just talk about what we'redoing, what we're finding is
helpful, where it's workingwell, where we still have to
exercise a lot of care.
Yeah, definitely.
Let's see.
(01:29):
So and we've we've done severalthings recently from all the way
from uh you know automating, notnecessarily automating, but
vibecoding a uh API notebookthat that pulls data out of out
of a source, um, which has beeninteresting.
We've you know created syntheticdata sets and demos, demo data
sets, because one of thestruggles is you want a data set
(01:52):
that replicates a business, butit you also want one that's
updating and new.
Yeah.
Um so you can kind of createyour own now without a ton of
effort, which has been reallycool.
Um Yeah, several more that I'mprobably not thinking of.
It is cool.
So you you manage our platformteam, and so you're you and your
guys manage all of our clients'data lake houses, but a big part
(02:14):
of that is pulling in new datasources.
So creating uh you know a reallyrobust data pipeline that can
run daily or multiple times aday and has a graceful fail mode
if something goes wrong so itdoesn't leave the data lake in
bad shape and sets off alertsand you know makes it easy for
the platform engineers, well,easier, to go in and fix it.
(02:37):
But also right up front, when anew data source comes from one
of our clients, we've got tomake that connection.
So we've got a series of Jupyternotebooks, depending on what
type of data source it is, so wecan make that connection.
In some cases, um, I think Idon't know, somewhere like a
hundred data sources.
We have something we've usedbefore, so we at least something
have something we can startwith.
(02:57):
Um sometimes we get brand newstuff though, like uh like a
totally novel API.
It's not uncommon for APIdocumentation to not be great.
Yeah.
And so, yeah, that's a perfectplace to use vibe coding.
Um, but we want to start withour standards.
We have best practices, we wantto do all those things I was
talking about.
So how do you how do you start?
(03:18):
How do you get uh your I don't Ithink you guys are using VS Code
with Claude.
Yeah, if I'm if I recall.
So how do you get it to knowwhat your target architecture is
and make sure it does a good joband doesn't go off and do
something kind of crazy?
Yeah, yeah.
So it's still a work inprogress.
We're still refining it, whichI'm sure all things AI are going
to be that way.
(03:39):
But um right now we're usingskills.
So we've you know kind oftrained skills to uh learn what
our current best practicetemplate is.
So what some of our best Pythonengineers made.
Um we gave it that and we said,hey, this is the stuff you need
to follow.
Here are the things youshouldn't do, here are the
things you should do.
Um it's actually quiteinteresting because we we ended
(04:01):
up trying to put everything inone skill.
And that so that includes likehow do I handle all these edge
cases?
How do I go out and finddocumentation?
What do I do if I can't find thedocumentation?
Yeah.
Okay.
Yeah.
And we found some weird behaviorwith that.
It's almost like we just gave ittoo much.
Yeah.
So one of the things we'reexperimenting with now, which
(04:24):
seems a little bit morepromising, is having discrete
skills that can call each other.
Oh, that's okay.
Do you have like an orchestratorskill that knows which skills to
call?
So, like, for example, I'mimagining you can tell me if
this is what you're doing ornot, but um you have a new API,
you can have a skill that goesout and reads the API
documentation.
Exactly.
For example, that's actuallywhat what led us to the idea
(04:47):
because we are having a lot oflot of trouble with the API
documentation, because not alldocumentation is AI friendly.
I just can't read it becausethey put it in JavaScript or
there's some weird backend onit.
So Yeah, interesting.
And some of these APIs have sortof secret off-the-menu stuff
that you have to know to reallymake it work well.
Yeah.
(05:07):
Yeah.
Yeah.
And then sometimes the APIsdon't work well, so you end up
going to the old ODBC orsomething.
Yeah.
Anyway, yeah.
So um I know you've got goalsaround bringing down the amount
of time it'll take to create oneof those.
One of the interesting things uhto me that uh you told me you
found out was that it's notnecessarily the hands-on
keyboard time, but there's a lotof time around that, getting
(05:29):
access to the data source,getting permissions set up
right, getting the OAuth set up,you know, uh the system
accounts, whatever you need, umuh as well as you know, looking
at code and and doing reviewsand all that.
So the hands-on keyboard timecan come down significantly, but
you can still end up spendingdays.
Like may maybe hands-on keyboardis half a day now.
(05:50):
Yeah.
But you can still spend days oreven a couple weeks getting it
done because of all thatancillary stuff.
Yeah.
And I sort of wonder if that'sgoing to be the case with a lot
of business agentics stuff for awhile.
Yeah.
I think it probably will,actually.
Yeah, I would I wouldn't besurprised.
You know, it's whenever you needto coordinate people, really.
That's where a lot of time a lotof times our roadblocks come
(06:12):
from.
So Yeah.
Yeah, it's true.
It's gonna it's gonna beinteresting to see how all of
that plays out.
Like if you tried to build askill that said, okay, contact
the client, get them to set upthe author, it's just not gonna
work because you need to talkthrough that very uh variable
thing that happens, I think.
Yeah, yeah.
And it's you know, we we've seenjust hundreds and hundreds and
(06:35):
hundreds of sources, and it'sjust always so completely
different.
Yeah.
You know, sometimes there'snothing even online about it,
right?
Yeah.
We've run into that before.
People just don't talk about itonline.
So the AI doesn't know, doesn'tisn't trained on it, it doesn't
know where to go find anythingabout it.
Kind of runs into a dead end.
Yeah.
Do you think does Claude Code doa pretty good job of following
(06:56):
our best practices?
It's I don't know.
I'd say like 80-20.
I might my team might say I'mbeing too generous to Claude.
Really?
Yeah.
Um they uh they have some oftheir times where it's like, oh
well, it just went over here andedited that thing that I didn't
want it to edit, right?
Right.
And so that's part of one of thereasons why we actually switched
(07:18):
to VS Code.
Um from just using the terminal.
Yeah, exactly.
Because one nice thing is thatas long as you don't tell it,
yeah, just go crazy, editwhatever the heck you want.
Um, you say, Let me approveedits, it will show you a
side-by-side comparison.
Here's your version, here's whatI'm gonna change, and it
highlights the changes for you.
You know, like those of you whomight have used um Git compare.
(07:41):
It's that essentially you'recomparing what the AI did to
what you have.
Yeah.
And so makes it way easier tomake sure it doesn't go off the
rails and edit somethingcompletely different, along with
sneaking in, you know, theediting you asked for.
I hate that.
Oh, and that was the problem,you know, six months, nine
months ago, whenever I wouldplay with vibe coding, because I
have been for, I don't know, thelast year or so.
(08:02):
But not that consistently, maybefor the first half of last year,
till we got up to the holidayseason, because it just wasn't
that good.
Yeah.
You know, it would producesomething that was close, and
then you'd risk r request fixes,and it would maybe make the fix
but break something else.
And it was just sort of it waslike playing whack-a-mole.
And then all of a sudden, aroundThanksgiving, Christmas time, it
(08:23):
stopped feeling like that.
And then around February, itreally stopped feeling like
that.
Yeah.
Yeah.
It's gotten so much better.
Yeah.
So uh well, you're gonna have totry GPT 5.5, the codex.
Yeah.
Uh and uh and let me know howthat is.
I'll be playing with that too.
Another thing I know you wereworking on, so we've got I'm
doing work for clients toconnect Claude and other um
(08:47):
agents to a data lakehouse, totheir data lake house, and be
able to do ad hoc queries forexecutives to be able to do, you
know, get quick answers onfinancial data or operational
data.
And it seems like that would beeasy, but um it's easy to get
wrong answers, actually.
And so we've been buildingsomething called a platinum
layer, which is to take the goldlayer out of the medallion
(09:09):
architecture and then uh do somethings to it, denormalize it
even more, rename columns,rename tables, make sure all of
our joins are done in aconsistent manner, and then
actually stack a bunch ofmarkdown files into the data
lake as well with context.
But then in addition to that,um, we started building custom
MCP servers for differentclients that that have a lot
(09:31):
more instructions about how togo and get an answer for the
agent.
Um, it's been reallyinteresting, and you've been
doing some work to build a demofor that.
Yeah.
And part of that was sort ofvibe coding a data set, right?
Because we don't want to use ourdata or any of our clients'
data, obviously, um to demo.
So we had to come up withcompletely synthetic data.
(09:52):
And there's things like umAdventureWorks out there or
Contoso or whatever, you know,the Microsoft thing.
So there's there's a lot ofstuff.
But but you actually startedfrom scratch.
Why don't you talk about that alittle bit?
I thought it was pretty cool.
Yeah, definitely.
So as the as the idea to startfrom scratch actually came from
AI itself.
Um, I I I just started asking itlike, how do we get around this
(10:13):
problem?
Help me brainstorm this, please.
Got that idea from it, and thenwas like, yes, let's go for it.
Um so I basically vibe codedit's it's nothing crazy, but
it's a it's a SQL script thatbuilds out tables, generates
random data, generates customersales reps, products, et cetera,
targets, um which then you justkind of run it.
(10:34):
It takes like 20 minutes becauseyou know, it it worked worked at
first.
And then where I got myself introuble is I started trying to
tweak it to try to make it seemmore real.
It's really hard to make likeseasonality.
Well, seasonality actually iseasy.
That's that's not hard.
Okay.
But like a customer leaving andthen maybe returning a year
(10:56):
later, right?
Stories like that, really,really hard to find.
Yeah.
Um, so I uh was trying to promptit to do that, and I ended up
destroying pretty much justwiping out the thing.
I mean, I didn't wreck itcompletely, but it made zero
sense.
Like I'd go ask, because I useClaude to connect to our MCP
server to talk to the data, andI'd be like, hey, tell me our
you know sales on this customerversus their target.
(11:19):
And it's like, well, they are2,000% above their target.
You should talk to the personwho set these targets.
I don't know what they werethinking.
Like, oh, oops.
So that was when I learned thatI really want Git to back up my
code base when I'm Oh, youweren't you didn't have that in
a repository?
I had to start from scratch.
Oh, that sucks.
(11:40):
Yeah, it was painful.
And we use we use uh AzureDevOps, right?
Yeah.
Are you s are you starting touse Git now that you're doing
more Vibe code?
I still use DevOps.
I mean it's Git on the end.
Um Yeah, I would like to getmore into Git, but we have we
have DevOps licenses.
It's kind of it's free for us,right?
So it is Git on DevOps.
So I just I guess I mean GitHub.
(12:01):
Yeah.
Yeah.
Yeah.
That's all my hobby stuff I'vebeen doing on that.
Yeah.
All right.
So then how did you get it?
So you started over, I guess.
Started from scratch,essentially.
Okay.
Pre-prompted, just just wipewiped it all out, started from
scratch, connected it to a repo,um, so I can commit changes and
roll back.
Yeah.
But yeah, it was a good learningexperience for sure.
(12:22):
Aaron Powell So one of theproblems with sample data sets
is that they're static usually,almost always.
And so as you're using them totest and so on, it's for data
that's six months old or twoyears old.
So have you figured out a way tosort of keep adding fresh
records?
Yeah, yeah, definitely.
So that was one of the keyrequirements I gave it.
You know, I that's one thingtoo.
(12:44):
Like when you Vive Code, yougotta start with really clear.
This is what it needs, this iswhat the end product needs to
look like.
Here are the things that itneeds to do, et cetera.
The clearer you can be, thebetter.
It created a pretty intense, Imean it's not intense in terms
of like sloppy, it's actuallyreally good uh procedure that
you call and it goes throughseveral different conditions.
(13:05):
It has seasonality built in andhas customer targets and
segments and kind of weightswhere their purchases go
depending on attributes on thecustomer.
Um and you can tell it, I wantto generate a thousand orders a
day, I want to generate 10orders a day, et cetera.
That's cool.
And so it was it was pretty coolto see because you know, with
Python, I immediately attachedto this is so easy to, you know,
(13:28):
like coding typical coding.
I know SQL's coding, but kind ofin a different bit.
Um immediately attached to howyou can do that.
With SQL, it's always been alittle harder to figure out how
do I get this thing to writegood SQL because you need that
data context of it.
Um data and you almost you haveto have a business context too,
right?
(13:48):
Exactly.
So you know, here's here's theconditions I'm trying to
simulate so that I can have aninteresting demo.
Yeah, exactly.
But basically it it's uh youknow, I consider myself a really
strong SQL developer.
I've been doing it for years andyears.
You are truly an expert, yeah.
And yeah, the stuff in there waswas insane, you know.
(14:08):
It was stuff that I knew, butthe way it put it all together
and in creative ways was wasreally cool to see.
Oh, that's cool.
How much debugging did you haveto do?
On that proc?
Yeah.
Yeah.
I went back and forth withClaude.
I don't have a number, but likeprobably three hours or so back
and forth.
But the thing is, you know, I'dhave to go go in there, then I'd
(14:29):
have to run everything, um, waitfor it to populate, and then
check the results.
Sometimes I'd export CSVs andsend it back to Claude.
I'd ask it like, hey, write me aquery that will summarize this
so we can make sure this makessense.
And I just paste the query in,you know, and run it.
Did you have it create a test uhtest suite to run?
I did not for that onespecifically, mainly because I
(14:52):
need to still figure out how toconnect AI to Well, I guess we
do know how to do it now.
To the data later.
Which right now, yes, I can dothat.
So that's a great idea.
Next time I'm gonna have it justrun its own tests, get its own
data back, adjust it.
Yeah.
It's a great idea.
So what's next for you with theteam here in terms of uh AI
(15:15):
coding?
Yeah, so you know, we're justtrying to get as efficient as we
possibly can with uh ourday-to-day work, right?
So a couple of things, you know,we're redesigning our standard
templates, um, we're making it abit more modular so that AI has
less to edit.
Right.
Okay.
Um there's a couple of drivingforces on that.
(15:36):
You know, obviously the less weneed to edit, the less context
it has to use on it.
Yeah.
Um more targeted it can be.
And then also, you know, ifprices increase like crazy for
some reason, we're not we're notcompletely underwater.
Right.
Um and then the other one that Ireally want to do too that I've
heard success about is umgetting just automated bug
(15:58):
fixes, right?
So pipeline for a client failsovernight, it's going to
automatically go look at thecode in our repos, um, make what
it thinks is the fix, put a pushuh pull request in, and then in
the morning we just look at it,right?
We'll still want that human inthe loop for for now.
Um that's pretty cool.
But I know some you can do acode review too with a
(16:20):
recommended, you know, I've I'vereviewed the pull request.
Yeah.
I recommend that this one beapproved.
Here's why.
Yeah you still might you stillmight look at it, but it might
speed you up.
Yeah, if we got trusty enoughwith it, yeah, maybe we have a
reviewer that reviews all therequests.
Yeah.
If it's ends up being goodenough, it can just
automatically push it and rerunit.
I'd be crazy.
I think these tools are gettingreally good on coding where you
(16:42):
can almost get there.
I mean you have to obviously besuper careful around anything
that has to do with security,exactly.
Or that's super important umoperational data.
You don't want to put bad datain the data lake in the lake
house.
Um but because it's funny, I wasreading an article about why do
LLMs hallucinate so much just infree text conversations or
(17:03):
summarizing dictation orwhatever, but in code it really
nails it.
And and some of the theorieswere that, well, there's so much
code out there that you cantrain way, way more than sort of
random um prose that's outthere.
And it's way more structured.
And the languages are veryconfined, like the the total
(17:24):
vocabulary and the syntax is notthat big.
So they can get really good atit.
And so it's not like the problemthey're seeing in law where you
know 75% of big legal briefswritten with uh with AI have
hallucinated stuff in them,made-up cases.
And I mean it just happenedagain to another big law firm,
amazingly.
(17:45):
Um it's it's different thanthat.
Not that it can't do thingswrong or start going down a bad
path or introduce some some bug,but it's amazing how good it's
gotten this last, I don't know,six months.
Yeah.
Just as really complete stepchange.
Oh, yeah, I completely agree.
I mean, it's got public reposout there all over the place.
Stack Overflow is a gold mine ofinfo now.
(18:07):
Yeah.
Yeah.
I've heard they're actually umselling their data now as
training data.
Oh, that's why models are that'swhere you used to go.
If you had a question, that'swhere you went.
Yeah, exactly.
I used to go there every singleday.
I'd always have Stack Overflowopen.
I haven't opened it in four orfive, probably more than that,
probably like six months at thispoint.
Yeah, I'm not surprised.
(18:28):
I just get answers faster, youknow, less digging.
I think next time I'd love totalk about sort of culturally
what it's been like to get theteam on board with that because
there's a there's a wide rangeof reactions to from oh, I can't
wait to do all write all my codethat way to gosh, one of the
things I really enjoy about myjob is crafting code.
Like it's almost an art.
(18:48):
So I don't really want to givethat up.
So anyway, we'll talk about thatnext time.
Yeah, absolutely.
But uh good discussion and uhtalk to you soon.
Yeah, thank you.
Appreciate it.
Talk to you later.
Popular Podcasts
Hey Jonas!
Hey Jonas! The official Jonas Brothers podcast. Hosted by Kevin, Joe, and Nick Jonas. It’s the Jonas Brothers you know... musicians, actors, and well, yes, brothers. Now, they’re sharing another side of themselves in the playful, intimate, and irreverent way only they can. Spend time with the Jonas Brothers here and stay a little bit longer for deep conversations like never before.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Las Culturistas with Matt Rogers and Bowen Yang
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.