April 13, 2025 • 25 mins
What if you could create a fully functional app just by expressing your idea in natural language?
In this episode, host Andreas Welsch catches up with Matt van Itallie, the founder and CEO of Sema, to explore the exciting world of vibecoding. Discover how vibecoding is democratizing app development, enabling even those without programming skills to prototype live applications with ease.
Matt shares valuable insights into the risks of relying too heavily on AI-generated code, emphasizing the importance of having a human in the loop to ensure security and quality:
- What is vibecoding, and why is it such a popular trend?
- How does vibecoding democratize access to software development?
- What are the unexpected challenges that novice coders run into?
- Which questions do investors and private equity firms ask before acquiring or investing in a software company?
Learn how to harness this powerful tool while avoiding common pitfalls, and understand what it means for developers and investors alike.
Don’t miss this chance to future-proof your business with actionable AI strategies—tune in now!
Questions or suggestions? Send me a Text Message.
***********
Disclaimer: Views are the participants’ own and do not represent those of any participant’s past, present, or future employers. Participation in this event is independent of any potential business relationship (past, present, or future) between the participants or between their employers.
Level up your AI Leadership game with the AI Leadership Handbook:
https://www.aileadershiphandbook.com
More details:
https://www.intelligence-briefing.com
All episodes:
https://www.intelligence-briefing.com/podcast
Get a weekly thought-provoking post in your inbox:
https://www.intelligence-briefing.com/newsletter
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Andreas Welsch (00:00):
Today, we'll talk about vibecoding and its
impact for developers andinvestors, and who better to
talk about it than someone who'sactively working on that, matt
van Itallie.
Hey, Matt.
Thank you so much for joining.
Matt van Itallie (00:11):
Thank you so much for having me.
I'm really excited to be here.
Andreas Welsch (00:14):
Wonderful.
Hey, why don't you tell ouraudience a little bit about
yourself, who you are and whatyou do.
Matt van Itallie (00:19):
Sure.
So I am the founder and CEO of asoftware company called Sema.
And what we do is bringexecutive insight into
engineering, demystifyengineering, software
development for non-technicalaudiences on the C-suite and the
board of directors.
I'm the son of a math teacherand a computer programmer, so I
(00:41):
was born to treat code as data.
And have had the great pleasurethrough sema to help analyze
more than a trillion and a halfworth of code bases over the
last eight years.
Andreas Welsch (00:54):
Wow.
That sounds impressive.
So I'm super excited to have youon.
Should we play a little game tokick things off?
Matt van Itallie (01:01):
Absolutely.
Let's
Andreas Welsch (01:02):
do it.
Okay.
What do you think?
If AI were a band, who or whatwould it be?
Matt van Itallie (01:11):
If AI were a band?
Who or what would it be?
It's probably a cop out to sayelectronica electronica who are
the folks who do whip it?
Is it Devo?
A normal but also kooky.
That that is how I think aboutabout AI.
(01:33):
It is human-like, but also kookyin some and surprising ways.
Andreas Welsch (01:37):
I love that.
That's a great answer.
I'm always surprised with whatmy guests come up with when I
ask these questions and what theanalogies are, but that's
definitely a great one.
It seems that the, internetlately has gone pretty crazy in
the last couple of weeks about anew trend, right?
And lots of people are havinggood time with that too,
vibecoding, and with a fewprompts and the right tool,
(01:59):
anyone can quickly build an appand pretty much any app too,
right?
All you need is an idea and anAI tool that can generate the
quote for you.
And now I know you've been inthis space for code analysis
and, these things for a numberof years, like you said.
I'm just curious, what are youmaking of this trend as somebody
(02:19):
who professionally looks atthese, type of things and, helps
companies understand how good istheir code and is it usable and
is it modern and all these kindof things.
Matt van Itallie (02:29):
Exactly.
I always like to start with the,personal take.
I, myself have been trying outvibecoding and have used it to
build simple prototypes ofapplications over the last few
months.
And I absolutely understand whatthe buzz is about, if I can.
(02:51):
I am pretty technical, butdefinitely not a professional
software developer.
And I am able using these vibecoding techniques to take an
idea heavily refine it first,and then produce a working code
that is live.
Just through my own efforts.
(03:13):
It's personally incrediblyexciting to be able to build
that way, to have outputs thatare accessible accessible to me
that weren't accessible before.
And it's also really helpful toour business, to be able to
really the, next level ofprototyping by showing real
(03:35):
products in the wild, so tospeak.
It makes come alive what we'rebuilding next, much more so than
ideas, words on a page or imageseven just a clickable prototype,
but a real a real live appreally goes a long way.
Andreas Welsch (03:54):
I have a couple follow ups for you in a second,
specifically around prototypingand, I've been experimenting
with white coding as well, andspend a couple long nights,
short nights, depending on howyou look at it.
Going down the rabbit hole ofgetting excited about something
that I built in a couple ofminutes and then spending hours
trying to get it to where Ireally want it to be.
(04:14):
And I'm, just amazed and excitedby the opportunity too.
You know me, too.
I've learned about softwaredevelopment in, college.
I did some, scripting when I wasin a, in an IT organization.
Also, I've never really been aprofessional software developer.
Like you shared this as well,but I know enough how how you
(04:36):
write code what, the differentelements and functions and
variables and constants andthings are and what they do.
So being able to, have thatbooster and, do that much more
quickly, I I can only share theexcitement.
It's, amazing.
These days.
But I'm curious you, work with alot of startups and founders
(04:57):
probably among many otherdifferent kinds of companies,
and who, are you seeing who isvibe coding actually for and to,
to what degree does it.
Democratize that access tosoftware development.
Matt van Itallie (05:09):
Yeah.
I use the very narrow term ofvibe coding to mean only using
human language based techniquesin order to produce code and
only edit it through the humanlanguage.
Interface, which is a particularsubset, a very important and
(05:31):
obviously very sexy subset ofthe overall approach of GenAI
enabled of GenAI enabledsoftware development, which
involves not only perhapsstarting with human language
interface, but then usingprofessional software
development techniques tomodify.
(05:52):
In the strictest definitionthere are many there's
absolutely right ademocratization of many folks
who are non non coders who areactually trying to build full
and complete applications justthrough vibe coding.
I believe the, latest from YCombinators, it may be a 10% of
their products, if I understandit right, of their recent class,
(06:14):
are almost 95%.
GenAI generated only code andthey're able to generate some
meaningful revenue from it.
We also know that inside productmanagement functions within
organizations that vibe codinghas has skyrocketed, which makes
(06:35):
sense.
Traditionally product managerswould build very detailed
requirements and then turn themover to an engineering team.
For prototyping, now forprototypes, and we're gonna talk
about the risks and why thelimitations of vibecoding, but
certainly for a prototype thatjust needs to get the idea
across soft excuse me.
(06:56):
Product managers are able to dothat themselves and skip a step.
And what I would say, thinkingabout how busy.
Our developers are take, removesome of the lower priority work.
Basically engineering, softwareengineering is so, valuable and
what they do is so valuable.
(07:16):
Anything we can do to takenon-essential work away from
them and let them focus on thehard stuff, the meaningful
stuff.
It's good for them, it's goodfor the organization, it's good
for ultimately users.
So I know from our experienceand more broadly product
management is probably the placewhere vibe coating has taken off
the most significantly at, theenterprise level.
Andreas Welsch (07:38):
That's exciting and still also a little bit
concerning.
I could imagine for softwaredevelopers who say if anybody
can do what I have learned andtrained, or maybe even they
think they can do it, but theyactually can't because I have
spent years honing my craft.
Where do you see this, going?
There's certainly leaders outthere, leaders of AI labs who
(08:01):
say.
By, a certain point in time inthe not too distant future, we
will only be using AI to codeand software development is
losing in relevance.
What do you make of that?
Matt van Itallie (08:16):
I studied history in undergrad, and I take
a longer term historical viewhere, which is let's say 70
years ago to be a computerprogrammer meant to manage.
Punch cards and keep track ofthem and make sure they were
sorted, and then feed them intothe computer.
And you were a computerprogrammer and you literally
were working on these cards.
(08:37):
And since came in no particularorder, version control systems
and IDs, like workspaces wherefolks can get work done and open
source code where you're pullingwhat you want to build, but you
get, you don't have to build ityourself.
And so in that sense the historyof software.
Development is one of greaterand greater abstraction where
(09:00):
you know the idea and then youhave more and more adept tools
to be able to carry this outeven before.
Even before the rise of AI forcoding if you take the, ratio of
software developers working onthe Apollo mission, so the space
control, NASA space controlsystems in the seventies, and
(09:21):
you compare that to the numberof software engineers working at
SpaceX, something like athousand times more productive
over the history over that, timeperiod of what individual coders
are able to do.
Now AI is a continuation ofthat.
It obviously, we all believethat it's a, greater
acceleration of what, ispossible.
(09:43):
But I think the nuance the theunderstanding of how to manage
edge cases, of how to managecomplex system, of how to
handle.
Existing code.
It's, different perhaps ifyou're starting with, new code
and how far you can get, but totake existing systems that are
powering everything we do sincesoftware has been eating the
(10:05):
world for some time developers.
Developers will remain extremelycritical in managing enterprise
systems.
They'll be empowered by AI, butthey'll remain absolutely
critical.
Andreas Welsch (10:22):
Something that I've experienced firsthand.
I feel it aligns pretty well tosome of the research in other
areas when it comes to GenAIuse, where some of those
sentiments where AI helps thosethat have a certain level of
proficiency be even moreproductive.
But it also helps those who aremore at an entry level get to a
similar level as, those thatare, proficient.
(10:45):
For me, when I vibe coded mybusiness simulation app, I felt
like it was propelling me and,accelerating parts of what I
know because I know where someof the gaps are, where some of
the issues are, or when I raninto issues that I could
troubleshoot it or that I had anidea what to look for and see
(11:06):
how I can tweak it.
So I think you absolutely have apoint there that if you are a
professional software developer,this helps you on one hand,
accelerate.
What do you do?
But on the other hand, you needto have that knowledge so you
can also either counteract orfigure out why it's not working
exactly as you want it to.
And guided, too.
It's almost like pottery.
Not sure how many softwaredevelopers will like that
(11:28):
statement, but we'll figure thatout in the comments.
Now.
You mentioned prototyping,right?
It helps you if you're inproduct management or maybe if,
you want to create a sales demoor just show the art of the
possible to some extent, butI'm, Concerned that people might
take this a step further and sayit's not just the art of the
(11:50):
possible.
It is the possible and we deployit and we maybe monetize it.
We put it out in the open.
We have a new app but maybe havenot applied the, same rigor that
professional software developerstoday would apply.
What unexpected challenges doyou see that novice coders might
run into?
Matt van Itallie (12:09):
Yeah, so there are, four primary risks for
using GenAI to code, whetherit's.
Vibe coding, and it's purelypurely using natural language or
using GenAI as a start.
And then also coding directlythe risks are greater with pure
(12:30):
vibe coding for sure than GenAIenabled or enhanced coding.
The first is security risk GenAIcode.
Leaves behind or introducessecurity vulnerabilities that
lets bad actors act orinformation leak out by the way,
so does all kinds of code.
So does human written code, sodoes open source code.
(12:50):
The point is not that one waynecessarily is more riskier than
others, but that they all need ahuman in the loop.
They need a review in order tomanage that risk.
A second is code quality andmaintainability and accuracy.
Again, a prototype might have asimple number of use cases, but
(13:12):
to get the edge cases right, andif you're gonna use it.
In an enterprise setting, forsure.
It needs to be exactly right.
Getting it close really doesn'tcut it.
It really doesn't cut it.
And coming along with that,there's such an increase in
volume of code because it's soeasy to just have the system
spit out spit out so much codethat it, it becomes harder, it's
(13:35):
harder to understand andtherefore harder to maintain.
Then there's two other issuesthat are at least in, in
specialized situations.
The third is GenAI code willunder certain circumstances,
likely not get copyrightprotection a form of
intellectual propertyprotection.
And so for enterpriseorganizations who copyright
(13:56):
their code, this isn't aboutcopyright infringement, but
copyright protection, it seemsincreasingly clear that they're
gonna have to change theirapproach to ip.
Management or and or emphasizethe importance of of software
developer involvement in thecode itself.
If you're listening to this andyou work at an enterprise that
copyrights code, please go talkto your lawyers to make sure
(14:19):
that they're helping you on a goforward plan.
And the fourth and final reasonis something we call exit risk.
And what exit risk is thepossibility that if you're
trying to sell your businessthat organizations who are
conducting technical duediligence, which is say looking
under the hood at the codeitself might conclude if, your
(14:42):
code was entirely vibe coded orGenAI generated, what are we
buying?
We can, just produce thisourselves.
Audience think about, theanalogy breaks down a little
bit, but think about a aprofessor assigning a term paper
and the student comes back andthe professor says, how much did
you do?
And the student said I just fedit into an LLM and spit out the
(15:05):
answer.
That's of course, unethical.
Although, I guess good, on youfor telling the truth.
This this student but itcertainly does not in any way
suggest there is intellectualknowledge in the underlying
document.
And that's the that's the veryreal risk.
In, diligence situations.
And by the way it's, alreadybegun to happen where folks are
walking away, organizations arewalking away from deals where
(15:29):
walking away from deals wherethe code is almost entirely
GenAI generated.
Andreas Welsch (15:36):
And, what are some of the reasons why they're
walking away?
Is it, because there's no actualIP in the business?
Matt van Itallie (15:44):
Yeah.
There's no IP if you can promptyour way to code, nothing's
stopping the buyer from buildingit themselves.
Andreas Welsch (15:51):
Yeah.
Now even before vibe coding wasa thing.
I think investors, PE firms, Iknow you've worked with some of
the largest PE firms on theplanet.
They started to get interestedin this composition.
How much is actually humancoded?
How much is AI coded?
What are some of the otherreasons you mentioned IP as one
(16:14):
part but why should it becareful there on one hand vs.
vibe coding, but have the humanin the loop.
Matt van Itallie (16:22):
Yeah, exactly.
So human, the humans in theloop.
Speaking of another reason whydevelopers remain so essential
is the best way to address allfour of those risks.
On the security risk side thereis in our view a presumption
that if the GenAI code has notbeen modified well, that means
it has not gone through asecurity review.
(16:43):
It's not feasible.
Code, it comes straight outtathe LLM to be security risk
free.
So it's a, there's a defaultpresumption that that sellers
would have to that would have tojustify.
And so having humans in loop andobviously involved in modifying
or blending that code instead ofit pure is what we would say is
at least a, is a signal thatthey they're addressing security
(17:06):
warnings quality andmaintainability.
Same thing if it just came.
Out of LLM, there's apresumption that it's definite
that it hasn't been deeplyunderstood.
And the edge case is added.
So the human blending reallymakes a difference on exit risk.
(17:26):
Huge difference between on exitrisk.
Huge difference between codethat started with GenAI and was
modified, and which is to saypure GenAI pure GenAI becoming
blended GenAI and just GenAIcode itself.
That stayed pure.
From the beginning makes areally big difference in terms
(17:47):
of the likelihood of theintellectual property.
And then finally, on the on theintellectual property risk side,
the copyright side, this is notlegal advice.
You should most certainly talkto your lawyers if if your
organization seeks copyrightprotection.
But the US Copyright Officeannounced earlier this year
that, prompting is not enough toget that protection.
(18:08):
You need to have humans actuallycreating the work.
It's a fundamental component.
Human involvement is afundamental component of that IP
protection.
Andreas Welsch (18:17):
Thank you for sharing that and I really
appreciate how you're speakingwith depth and experience on
what you're seeing in themarket.
It sounds so easy.
Soon enough we won't be coding.
AI will do all, the work in somany different sectors and
professions.
Software development is as well.
But then when you look under thehood and like you just did and
(18:39):
you said IP security, exit riskand so on, those are the sort
real risks, especially if youwant to sell your business at
some point or you want to sellyour app to, to an investor.
We've already covered a lot ofground from by coding as this
cool new trend.
You don't have to be aprogrammer.
(18:59):
You can just instruct the LLM orthe, tool to create an app for
you.
Using natural language, you canrefine it.
You either should know whatyou're doing is what I'm
hearing, or you should knowsomebody who can help you.
Figure out what you're doingand, that you're doing it right
so that you're protected furtherdown the line.
(19:21):
What are some of the otherthings that you're hearing in,
in, in the industry when itcomes to coding GenAI coding?
Especially as far as ininvestors, PE firms, private
equities are, concerned.
What's top of mind for them?
Matt van Itallie (19:35):
Yeah.
I'm I'm happy to share with youand your audience the results of
of our benchmarking, which webelieve is some of the, first
benchmarking there is on a GenAg code usage, certainly among
investor backed companies.
And what we found at EnterpriseGrade Code by large part
investor backed companies, GenAIis approximate for the median
(20:00):
organization.
I'll just get it right.
Organizations are using GenAI tostart.
About 11% of their code, theirproduction code.
That's the 50th percentile.
The 25th percentile is 6% andthe 75th percentile is 16%.
So half the organizations are atthe, between starting GenAI
(20:21):
between 11, excuse me, betweensix and 16%.
The pure GenAI code, where itstarted with GenAI but has not
been modified is at most.
Between three and 9% of thecode.
That to us, is a really goodsign because that means
everybody's getting the messagethat.
Starting with GenAI code is agood idea in at least some
(20:43):
circumstances which we mostcertainly agree with.
But my goodness, pure GenAI codeis, really risky and should be
kept kept to the minimum inenterprise settings like this.
So developers know it, they knowcode is a craft.
It is not just reducible tomechanical process and they know
(21:04):
that they have to be involved tomake sure it's correct and
building the right thing andbuilding it with care.
And also the organizations theyare a part of have gotten the
message that GenAI is a very,powerful tool and it's only
gonna grow in in in importancein, the developer's tool chain.
But my goodness, that needs tobe done with care care and
(21:26):
precision in in the productionstack.
And, when the code actually getsin the code that actually gets
used by enterprise users.
Andreas Welsch (21:35):
So the question that I wanted to ask earlier
goes in the same direction iswhen you hear, again, AI is
going to make coding that justeasier, but it's going to
automate it.
When you hear about the, riskson, one hand, right?
It's not as simple, it's not asstraightforward, especially not
if you want to patent it or ifyou want to copyright the work
(21:57):
that you're doing.
Take it with a grain of salt.
And on the other hand.
To your point the, data seemsto, show that it's already a
great starting point to buildupon, to refine it, to extend
it.
So good, happy medium as in manythings in life, right?
Matt van Itallie (22:15):
Exactly we say the goldilock zone.
Literally we explain to ourclients you too little GenAI
code.
And you are missing out on agreat opportunity for
productivity, for jobsatisfaction, for all the above,
but too much and certainly toomuch pure as opposed to blended
code.
And you're opening up theorganization some real risk.
Andreas Welsch (22:35):
Now, Matt, we're getting close to the end of the
show today and I was wonderingif you can summarize the key
three takeaways for our audiencetoday from our episode.
Matt van Itallie (22:43):
All right.
Certainly number one is that ifAI was a band, it would be Devo.
Wait, that doesn't count.
Number one, everyone should betrying everyone should be trying
vibe coding, and if that feelslike too much, just try vibe
writing.
Have it produce an article foryou in human language.
You just, everything we're doingthese days is is everything
(23:06):
we're doing electronically, atleast is going to be AI enabled,
if not AI base.
And the faster you can wrap yourhead around it, whether it's
picking a movie or or doingsomething at work, the easier
you'll be able to understand itscapabilities and its limits.
That could be through vibecoding, if you wanted to build
something or see what it's like,give it a shot.
But if not, pick something andput it to work.
(23:29):
Everyone has to be an AIenabled.
There's, you'd just be massivelymissing out.
Again, whether it's vibe codingor not.
Second vibe coding in particularis a spectacular way for to
enable prototyping andespecially to help enable
product management teams get totesting faster, in particular,
by protecting engineers' time.
(23:51):
And third, GenAI coding ingeneral and vibe coding in
particular is incrediblydangerous in enterprise
settings.
And it must to mitigate musthave a human in the loop, must
have developers blending thatGenAI code.
Pure GenAI code is extremelyrisky.
And our view should not be usedin enterprise settings beyond a
(24:14):
di minimis amount.
Andreas Welsch (24:16):
Thank you so much, Matt.
It's been a pleasure having youon and learning from you and,
hearing all these insights fromyou on how you can use GenAI as
a developer and also what itmeans if you're an investor, if
you're trying to sell your appto an investor, things that we
should all be aware of inenterprise especially.
Matt, thank you so much forjoining us.
Matt van Itallie (24:35):
Thanks so much for having me.
I really enjoyed it.
Andreas Welsch (24:37):
Wonderful.
Today, we'll talk about vibecoding and its
impact for developers andinvestors, and who better to
talk about it than someone who'sactively working on that, matt
van Itallie.
Hey, Matt.
Thank you so much for joining.
Matt van Itallie (00:11):
Thank you so much for having me.
I'm really excited to be here.
Andreas Welsch (00:14):
Wonderful.
Hey, why don't you tell ouraudience a little bit about
yourself, who you are and whatyou do.
Matt van Itallie (00:19):
Sure.
So I am the founder and CEO of asoftware company called Sema.
And what we do is bringexecutive insight into
engineering, demystifyengineering, software
development for non-technicalaudiences on the C-suite and the
board of directors.
I'm the son of a math teacherand a computer programmer, so I
(00:41):
was born to treat code as data.
And have had the great pleasurethrough sema to help analyze
more than a trillion and a halfworth of code bases over the
last eight years.
Andreas Welsch (00:54):
Wow.
That sounds impressive.
So I'm super excited to have youon.
Should we play a little game tokick things off?
Matt van Itallie (01:01):
Absolutely.
Let's
Andreas Welsch (01:02):
do it.
Okay.
What do you think?
If AI were a band, who or whatwould it be?
Matt van Itallie (01:11):
If AI were a band?
Who or what would it be?
It's probably a cop out to sayelectronica electronica who are
the folks who do whip it?
Is it Devo?
A normal but also kooky.
That that is how I think aboutabout AI.
(01:33):
It is human-like, but also kookyin some and surprising ways.
Andreas Welsch (01:37):
I love that.
That's a great answer.
I'm always surprised with whatmy guests come up with when I
ask these questions and what theanalogies are, but that's
definitely a great one.
It seems that the, internetlately has gone pretty crazy in
the last couple of weeks about anew trend, right?
And lots of people are havinggood time with that too,
vibecoding, and with a fewprompts and the right tool,
(01:59):
anyone can quickly build an appand pretty much any app too,
right?
All you need is an idea and anAI tool that can generate the
quote for you.
And now I know you've been inthis space for code analysis
and, these things for a numberof years, like you said.
I'm just curious, what are youmaking of this trend as somebody
(02:19):
who professionally looks atthese, type of things and, helps
companies understand how good istheir code and is it usable and
is it modern and all these kindof things.
Matt van Itallie (02:29):
Exactly.
I always like to start with the,personal take.
I, myself have been trying outvibecoding and have used it to
build simple prototypes ofapplications over the last few
months.
And I absolutely understand whatthe buzz is about, if I can.
(02:51):
I am pretty technical, butdefinitely not a professional
software developer.
And I am able using these vibecoding techniques to take an
idea heavily refine it first,and then produce a working code
that is live.
Just through my own efforts.
(03:13):
It's personally incrediblyexciting to be able to build
that way, to have outputs thatare accessible accessible to me
that weren't accessible before.
And it's also really helpful toour business, to be able to
really the, next level ofprototyping by showing real
(03:35):
products in the wild, so tospeak.
It makes come alive what we'rebuilding next, much more so than
ideas, words on a page or imageseven just a clickable prototype,
but a real a real live appreally goes a long way.
Andreas Welsch (03:54):
I have a couple follow ups for you in a second,
specifically around prototypingand, I've been experimenting
with white coding as well, andspend a couple long nights,
short nights, depending on howyou look at it.
Going down the rabbit hole ofgetting excited about something
that I built in a couple ofminutes and then spending hours
trying to get it to where Ireally want it to be.
(04:14):
And I'm, just amazed and excitedby the opportunity too.
You know me, too.
I've learned about softwaredevelopment in, college.
I did some, scripting when I wasin a, in an IT organization.
Also, I've never really been aprofessional software developer.
Like you shared this as well,but I know enough how how you
(04:36):
write code what, the differentelements and functions and
variables and constants andthings are and what they do.
So being able to, have thatbooster and, do that much more
quickly, I I can only share theexcitement.
It's, amazing.
These days.
But I'm curious you, work with alot of startups and founders
(04:57):
probably among many otherdifferent kinds of companies,
and who, are you seeing who isvibe coding actually for and to,
to what degree does it.
Democratize that access tosoftware development.
Matt van Itallie (05:09):
Yeah.
I use the very narrow term ofvibe coding to mean only using
human language based techniquesin order to produce code and
only edit it through the humanlanguage.
Interface, which is a particularsubset, a very important and
(05:31):
obviously very sexy subset ofthe overall approach of GenAI
enabled of GenAI enabledsoftware development, which
involves not only perhapsstarting with human language
interface, but then usingprofessional software
development techniques tomodify.
(05:52):
In the strictest definitionthere are many there's
absolutely right ademocratization of many folks
who are non non coders who areactually trying to build full
and complete applications justthrough vibe coding.
I believe the, latest from YCombinators, it may be a 10% of
their products, if I understandit right, of their recent class,
(06:14):
are almost 95%.
GenAI generated only code andthey're able to generate some
meaningful revenue from it.
We also know that inside productmanagement functions within
organizations that vibe codinghas has skyrocketed, which makes
(06:35):
sense.
Traditionally product managerswould build very detailed
requirements and then turn themover to an engineering team.
For prototyping, now forprototypes, and we're gonna talk
about the risks and why thelimitations of vibecoding, but
certainly for a prototype thatjust needs to get the idea
across soft excuse me.
(06:56):
Product managers are able to dothat themselves and skip a step.
And what I would say, thinkingabout how busy.
Our developers are take, removesome of the lower priority work.
Basically engineering, softwareengineering is so, valuable and
what they do is so valuable.
(07:16):
Anything we can do to takenon-essential work away from
them and let them focus on thehard stuff, the meaningful
stuff.
It's good for them, it's goodfor the organization, it's good
for ultimately users.
So I know from our experienceand more broadly product
management is probably the placewhere vibe coating has taken off
the most significantly at, theenterprise level.
Andreas Welsch (07:38):
That's exciting and still also a little bit
concerning.
I could imagine for softwaredevelopers who say if anybody
can do what I have learned andtrained, or maybe even they
think they can do it, but theyactually can't because I have
spent years honing my craft.
Where do you see this, going?
There's certainly leaders outthere, leaders of AI labs who
(08:01):
say.
By, a certain point in time inthe not too distant future, we
will only be using AI to codeand software development is
losing in relevance.
What do you make of that?
Matt van Itallie (08:16):
I studied history in undergrad, and I take
a longer term historical viewhere, which is let's say 70
years ago to be a computerprogrammer meant to manage.
Punch cards and keep track ofthem and make sure they were
sorted, and then feed them intothe computer.
And you were a computerprogrammer and you literally
were working on these cards.
(08:37):
And since came in no particularorder, version control systems
and IDs, like workspaces wherefolks can get work done and open
source code where you're pullingwhat you want to build, but you
get, you don't have to build ityourself.
And so in that sense the historyof software.
Development is one of greaterand greater abstraction where
(09:00):
you know the idea and then youhave more and more adept tools
to be able to carry this outeven before.
Even before the rise of AI forcoding if you take the, ratio of
software developers working onthe Apollo mission, so the space
control, NASA space controlsystems in the seventies, and
(09:21):
you compare that to the numberof software engineers working at
SpaceX, something like athousand times more productive
over the history over that, timeperiod of what individual coders
are able to do.
Now AI is a continuation ofthat.
It obviously, we all believethat it's a, greater
acceleration of what, ispossible.
(09:43):
But I think the nuance the theunderstanding of how to manage
edge cases, of how to managecomplex system, of how to
handle.
Existing code.
It's, different perhaps ifyou're starting with, new code
and how far you can get, but totake existing systems that are
powering everything we do sincesoftware has been eating the
(10:05):
world for some time developers.
Developers will remain extremelycritical in managing enterprise
systems.
They'll be empowered by AI, butthey'll remain absolutely
critical.
Andreas Welsch (10:22):
Something that I've experienced firsthand.
I feel it aligns pretty well tosome of the research in other
areas when it comes to GenAIuse, where some of those
sentiments where AI helps thosethat have a certain level of
proficiency be even moreproductive.
But it also helps those who aremore at an entry level get to a
similar level as, those thatare, proficient.
(10:45):
For me, when I vibe coded mybusiness simulation app, I felt
like it was propelling me and,accelerating parts of what I
know because I know where someof the gaps are, where some of
the issues are, or when I raninto issues that I could
troubleshoot it or that I had anidea what to look for and see
(11:06):
how I can tweak it.
So I think you absolutely have apoint there that if you are a
professional software developer,this helps you on one hand,
accelerate.
What do you do?
But on the other hand, you needto have that knowledge so you
can also either counteract orfigure out why it's not working
exactly as you want it to.
And guided, too.
It's almost like pottery.
Not sure how many softwaredevelopers will like that
(11:28):
statement, but we'll figure thatout in the comments.
Now.
You mentioned prototyping,right?
It helps you if you're inproduct management or maybe if,
you want to create a sales demoor just show the art of the
possible to some extent, butI'm, Concerned that people might
take this a step further and sayit's not just the art of the
(11:50):
possible.
It is the possible and we deployit and we maybe monetize it.
We put it out in the open.
We have a new app but maybe havenot applied the, same rigor that
professional software developerstoday would apply.
What unexpected challenges doyou see that novice coders might
run into?
Matt van Itallie (12:09):
Yeah, so there are, four primary risks for
using GenAI to code, whetherit's.
Vibe coding, and it's purelypurely using natural language or
using GenAI as a start.
And then also coding directlythe risks are greater with pure
(12:30):
vibe coding for sure than GenAIenabled or enhanced coding.
The first is security risk GenAIcode.
Leaves behind or introducessecurity vulnerabilities that
lets bad actors act orinformation leak out by the way,
so does all kinds of code.
So does human written code, sodoes open source code.
(12:50):
The point is not that one waynecessarily is more riskier than
others, but that they all need ahuman in the loop.
They need a review in order tomanage that risk.
A second is code quality andmaintainability and accuracy.
Again, a prototype might have asimple number of use cases, but
(13:12):
to get the edge cases right, andif you're gonna use it.
In an enterprise setting, forsure.
It needs to be exactly right.
Getting it close really doesn'tcut it.
It really doesn't cut it.
And coming along with that,there's such an increase in
volume of code because it's soeasy to just have the system
spit out spit out so much codethat it, it becomes harder, it's
(13:35):
harder to understand andtherefore harder to maintain.
Then there's two other issuesthat are at least in, in
specialized situations.
The third is GenAI code willunder certain circumstances,
likely not get copyrightprotection a form of
intellectual propertyprotection.
And so for enterpriseorganizations who copyright
(13:56):
their code, this isn't aboutcopyright infringement, but
copyright protection, it seemsincreasingly clear that they're
gonna have to change theirapproach to ip.
Management or and or emphasizethe importance of of software
developer involvement in thecode itself.
If you're listening to this andyou work at an enterprise that
copyrights code, please go talkto your lawyers to make sure
(14:19):
that they're helping you on a goforward plan.
And the fourth and final reasonis something we call exit risk.
And what exit risk is thepossibility that if you're
trying to sell your businessthat organizations who are
conducting technical duediligence, which is say looking
under the hood at the codeitself might conclude if, your
(14:42):
code was entirely vibe coded orGenAI generated, what are we
buying?
We can, just produce thisourselves.
Audience think about, theanalogy breaks down a little
bit, but think about a aprofessor assigning a term paper
and the student comes back andthe professor says, how much did
you do?
And the student said I just fedit into an LLM and spit out the
(15:05):
answer.
That's of course, unethical.
Although, I guess good, on youfor telling the truth.
This this student but itcertainly does not in any way
suggest there is intellectualknowledge in the underlying
document.
And that's the that's the veryreal risk.
In, diligence situations.
And by the way it's, alreadybegun to happen where folks are
walking away, organizations arewalking away from deals where
(15:29):
walking away from deals wherethe code is almost entirely
GenAI generated.
Andreas Welsch (15:36):
And, what are some of the reasons why they're
walking away?
Is it, because there's no actualIP in the business?
Matt van Itallie (15:44):
Yeah.
There's no IP if you can promptyour way to code, nothing's
stopping the buyer from buildingit themselves.
Andreas Welsch (15:51):
Yeah.
Now even before vibe coding wasa thing.
I think investors, PE firms, Iknow you've worked with some of
the largest PE firms on theplanet.
They started to get interestedin this composition.
How much is actually humancoded?
How much is AI coded?
What are some of the otherreasons you mentioned IP as one
(16:14):
part but why should it becareful there on one hand vs.
vibe coding, but have the humanin the loop.
Matt van Itallie (16:22):
Yeah, exactly.
So human, the humans in theloop.
Speaking of another reason whydevelopers remain so essential
is the best way to address allfour of those risks.
On the security risk side thereis in our view a presumption
that if the GenAI code has notbeen modified well, that means
it has not gone through asecurity review.
(16:43):
It's not feasible.
Code, it comes straight outtathe LLM to be security risk
free.
So it's a, there's a defaultpresumption that that sellers
would have to that would have tojustify.
And so having humans in loop andobviously involved in modifying
or blending that code instead ofit pure is what we would say is
at least a, is a signal thatthey they're addressing security
(17:06):
warnings quality andmaintainability.
Same thing if it just came.
Out of LLM, there's apresumption that it's definite
that it hasn't been deeplyunderstood.
And the edge case is added.
So the human blending reallymakes a difference on exit risk.
(17:26):
Huge difference between on exitrisk.
Huge difference between codethat started with GenAI and was
modified, and which is to saypure GenAI pure GenAI becoming
blended GenAI and just GenAIcode itself.
That stayed pure.
From the beginning makes areally big difference in terms
(17:47):
of the likelihood of theintellectual property.
And then finally, on the on theintellectual property risk side,
the copyright side, this is notlegal advice.
You should most certainly talkto your lawyers if if your
organization seeks copyrightprotection.
But the US Copyright Officeannounced earlier this year
that, prompting is not enough toget that protection.
(18:08):
You need to have humans actuallycreating the work.
It's a fundamental component.
Human involvement is afundamental component of that IP
protection.
Andreas Welsch (18:17):
Thank you for sharing that and I really
appreciate how you're speakingwith depth and experience on
what you're seeing in themarket.
It sounds so easy.
Soon enough we won't be coding.
AI will do all, the work in somany different sectors and
professions.
Software development is as well.
But then when you look under thehood and like you just did and
(18:39):
you said IP security, exit riskand so on, those are the sort
real risks, especially if youwant to sell your business at
some point or you want to sellyour app to, to an investor.
We've already covered a lot ofground from by coding as this
cool new trend.
You don't have to be aprogrammer.
(18:59):
You can just instruct the LLM orthe, tool to create an app for
you.
Using natural language, you canrefine it.
You either should know whatyou're doing is what I'm
hearing, or you should knowsomebody who can help you.
Figure out what you're doingand, that you're doing it right
so that you're protected furtherdown the line.
(19:21):
What are some of the otherthings that you're hearing in,
in, in the industry when itcomes to coding GenAI coding?
Especially as far as ininvestors, PE firms, private
equities are, concerned.
What's top of mind for them?
Matt van Itallie (19:35):
Yeah.
I'm I'm happy to share with youand your audience the results of
of our benchmarking, which webelieve is some of the, first
benchmarking there is on a GenAg code usage, certainly among
investor backed companies.
And what we found at EnterpriseGrade Code by large part
investor backed companies, GenAIis approximate for the median
(20:00):
organization.
I'll just get it right.
Organizations are using GenAI tostart.
About 11% of their code, theirproduction code.
That's the 50th percentile.
The 25th percentile is 6% andthe 75th percentile is 16%.
So half the organizations are atthe, between starting GenAI
(20:21):
between 11, excuse me, betweensix and 16%.
The pure GenAI code, where itstarted with GenAI but has not
been modified is at most.
Between three and 9% of thecode.
That to us, is a really goodsign because that means
everybody's getting the messagethat.
Starting with GenAI code is agood idea in at least some
(20:43):
circumstances which we mostcertainly agree with.
But my goodness, pure GenAI codeis, really risky and should be
kept kept to the minimum inenterprise settings like this.
So developers know it, they knowcode is a craft.
It is not just reducible tomechanical process and they know
(21:04):
that they have to be involved tomake sure it's correct and
building the right thing andbuilding it with care.
And also the organizations theyare a part of have gotten the
message that GenAI is a very,powerful tool and it's only
gonna grow in in in importancein, the developer's tool chain.
But my goodness, that needs tobe done with care care and
(21:26):
precision in in the productionstack.
And, when the code actually getsin the code that actually gets
used by enterprise users.
Andreas Welsch (21:35):
So the question that I wanted to ask earlier
goes in the same direction iswhen you hear, again, AI is
going to make coding that justeasier, but it's going to
automate it.
When you hear about the, riskson, one hand, right?
It's not as simple, it's not asstraightforward, especially not
if you want to patent it or ifyou want to copyright the work
(21:57):
that you're doing.
Take it with a grain of salt.
And on the other hand.
To your point the, data seemsto, show that it's already a
great starting point to buildupon, to refine it, to extend
it.
So good, happy medium as in manythings in life, right?
Matt van Itallie (22:15):
Exactly we say the goldilock zone.
Literally we explain to ourclients you too little GenAI
code.
And you are missing out on agreat opportunity for
productivity, for jobsatisfaction, for all the above,
but too much and certainly toomuch pure as opposed to blended
code.
And you're opening up theorganization some real risk.
Andreas Welsch (22:35):
Now, Matt, we're getting close to the end of the
show today and I was wonderingif you can summarize the key
three takeaways for our audiencetoday from our episode.
Matt van Itallie (22:43):
All right.
Certainly number one is that ifAI was a band, it would be Devo.
Wait, that doesn't count.
Number one, everyone should betrying everyone should be trying
vibe coding, and if that feelslike too much, just try vibe
writing.
Have it produce an article foryou in human language.
You just, everything we're doingthese days is is everything
(23:06):
we're doing electronically, atleast is going to be AI enabled,
if not AI base.
And the faster you can wrap yourhead around it, whether it's
picking a movie or or doingsomething at work, the easier
you'll be able to understand itscapabilities and its limits.
That could be through vibecoding, if you wanted to build
something or see what it's like,give it a shot.
But if not, pick something andput it to work.
(23:29):
Everyone has to be an AIenabled.
There's, you'd just be massivelymissing out.
Again, whether it's vibe codingor not.
Second vibe coding in particularis a spectacular way for to
enable prototyping andespecially to help enable
product management teams get totesting faster, in particular,
by protecting engineers' time.
(23:51):
And third, GenAI coding ingeneral and vibe coding in
particular is incrediblydangerous in enterprise
settings.
And it must to mitigate musthave a human in the loop, must
have developers blending thatGenAI code.
Pure GenAI code is extremelyrisky.
And our view should not be usedin enterprise settings beyond a
(24:14):
di minimis amount.
Andreas Welsch (24:16):
Thank you so much, Matt.
It's been a pleasure having youon and learning from you and,
hearing all these insights fromyou on how you can use GenAI as
a developer and also what itmeans if you're an investor, if
you're trying to sell your appto an investor, things that we
should all be aware of inenterprise especially.
Matt, thank you so much forjoining us.
Matt van Itallie (24:35):
Thanks so much for having me.
I really enjoyed it.
Andreas Welsch (24:37):
Wonderful.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Therapy Gecko
An unlicensed lizard psychologist travels the universe talking to strangers about absolutely nothing. TO CALL THE GECKO: follow me on https://www.twitch.tv/lyleforever to get a notification for when I am taking calls. I am usually live Mondays, Wednesdays, and Fridays but lately a lot of other times too. I am a gecko.