Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:02):
Media, Hello and welcomes a better offline. I'm, of course
your host ed Zitron. We're in the third episode of
our four part series where I give you a comprehensive
(00:23):
explanation as to the origins of the AI bubble, the
mythology sustaining it, and why it's destined to end really,
really badly. Now, if you're jumping in now, please start
from the very beginning. The reason why this is a
four part my first ever, is because I want it
to be comprehensive, and because this is a very big
subject with a lot of moving parts and even more bullshit.
(00:43):
A few weeks ago, I published a premium newsletter that
explained how everybody is losing money on generative AI, in
part because the costs of running AI models is increasing,
and in part because the software itself doesn't do enough
to warrant the costs associated with running them, which are
already subsidized and unprofitable for the model providers. Outside of
open and to a lesser extent, Anthropic, nobody seems to
be making much revenue, with the most successful company being
(01:05):
any Sphere, makers of AI coding tool Cursor, which hid
five hundred million dollars have annualized so forty one point
six million in one month a few months ago, just
before Anthropic and open ai jacked up the prices for
priority processing on enterprise queries, raising their operating costs as
a result. In any case, that's some pissport revenue for
an industry that's meant to be the future of software.
(01:26):
Smart Watchers are projected to make thirty two billion dollars
this year, and as I've mentioned in the past, the
Magnificent Seven expect to make thirty five billion dollars or
so in revenue from AI this year, and I think
in total, when you're throw in core even all them,
it's barely fifty five billion dollars in total. Even Anthropic
and open Ai seem a little lethargic, both burning billions
of dollars while making by my estimates, no more than
(01:47):
two billion dollars in Anthropics case this year so far
and six point six two six billion dollars in twenty
twenty five so far for open Ai, despite projections of
five billion dollars and thirteen billion dollars respectively. Outside of
these two AI startups are floundering, struggling to stay alive
and raising money in several hundred million dollar versus their
negative gross margin businesses flounder as they dug into. A
(02:09):
few months ago, I could find only twelve AI powered
companies making more than eight point three million dollars a month,
with two of them slightly improving their revenue, specifically AI
search company perplexd, which is now here one hundred and
fifty million dollars an ur in or twelve point five
million dollars a month, and AI coding startup Replayer, which
has hit the same amount. Both of these companies burn
ridiculous amounts of money. Paplexd burned one hundred and sixty
(02:32):
four percent of its revenue on Amazon web services, open
Ai and Anthropic last year, and while replet hasn't leaked
its costs, the information reports its gross margins in July
but twenty three percent, which doesn't include the cost of
its free users, which you simply have to do with llms,
as free users are capable of costing you a shit
ton of money. And some of you might say that's
how they do it in software, Well, guess what software
(02:54):
doesn't usually connect you to a model that can burn
I don't know ten cents twenty cents every time they
touch it, which may not seem like much, but when
you're making three dollars on someone and they don't convert,
it does problematically. Your paid users also cost you more
than they bring in as well. In fact, every user
loses you money in Generative AI because it's impossible to
(03:14):
do cost control in a consistent manner. A few months ago,
I did a piece of Anthropic losing money on every
single claud code subscriber. And now I'm going to walk
you through the whole story in a simplified fashion because
it's quite important. So claud Code is a coding environment
that people use used, or I should really say, try
to use to build software using generative AI. It's available
(03:36):
as part of Anthropics twenty dollars, one hundred dollars and
two hundred dollars a month claud subscriptions, with the more
expensive subscriptions having more generous rate limits. Generally, these subscriptions
are all you can eat. You can use them as
much as you want until you hit limits, rather than
paying for the actual tokens you burn. When I say
burn tokens and someone reached out saying I should specify this,
I'm describing how these models are traditionally built. In general,
(04:00):
you'll builded a dollar per million input tokens as in
user feeding in data and output tokens the output created,
so you wouldn't get one token built, so every million
you get charged. So, for example, Anthropic charges three dollars
per million input tokens and six million output tokens to
use its clauds on it for model, and it's about
I think, well, a word before tokens should really look
(04:23):
that up. It's it also gets more complex as you
get into things like generating code. Nevertheless, claud code has
been quite popular, and a user created a program called
cc usage which allowed you to see your token burn
the amount of tokens you were using. You were actually
burning using Anthropics models while using clawed code versus just
(04:43):
getting charged a month and not knowing, and many were
seeing that they were burning in the excess of their
monthly spend. To be clear, this is the token price
based on anthropics own pricing, and thus the cost of
Anthropic are likely not identical. So I got a little
clever using anthropics gross profit margins, I chose fifty five percent,
and then a few weeks solved my article sixty percent
was leaked. I found at least twenty different accounts of
(05:03):
people costing Anthropic anywhere from one hundred and thirty percent
to three thousand and eighty four percent of their subscription.
There is also now a leader board called vibrank, where
people compete to see how much they burn with the
current leader burning and I sheit you not fifty two
hundred and ninety one dollars of the course of a month.
Anthropic is, to be clear, the second largest model developer
(05:25):
and has some of the best AI talent in the industry.
It has a better handle on its infrastructure than anyone
outside of big tech and open AI, and it still
cannot seem to fix this problem even with weekly rate
limits brought in at the end of August. While one
could assume that Anthropic is simply letting users run wild,
my theory is far simpler. Even the model developers have
no real way of limiting user activity, likely due to
(05:47):
the architecture of generative AI. I know it sounds insane,
but at the most advanced level. Even there, modeled providers
are still prompting their models, and whatever rate limits may
be in place appear to at times get completely ignored,
and there doesn't seem to be anything they can do
to stop it now. Really, Anthropic counts amongst its capitalist
apex predators one lone Chinese man who spent fifty thousand
(06:09):
dollars to their compute in the space of a month
fucking around with glord code. Even if Anthropic was profitable,
it isn't, and we'll burn billions of dollars this year.
A customer paying two hundred dollars a month ran up
fifty thousand dollars in costs, immediately devouring the margin of
any user running the service that day, that week, or
even that month. Even if Anthropics costs are half the
(06:30):
published rates, they're not. By the way, one guy amounted
to one hundred and twenty five US is worth of
monthly revenue. This is not a real business. That's a
bad business without of control costs, and it doesn't appear
anybody has these costs under control and face with the
grim reality ahead of them, these companies are trying nasty
little tricks on their customers to douce more revenue from them.
(06:51):
A few weeks ago, Replet, an unprofitable AI coding company,
released a product called Agent three, which promised to be
ten times more autonomous and offer infinitely more possible abilities,
testing and fixing its code, constantly improving your application behind
the scenes in a reflection loop. Sounds very real, sounds
extremely real, It's so real, but actually it isn't. In reality.
(07:12):
This means you go and tell the model to build something,
and it would go and do it, and you'll be
shocked to hear that these models can't be relied upon
to go and do anything. Please note that this was
launched a few months after Replet raise their prices, shifting
to obfiscated effort based pricing that would charge the full
scope of the agent's work. And if you're wondering what
the fuck that means, so are their customers. Agent three
has been a disaster. Users found the tasks that previously
(07:35):
cost a few dollars were spiraling into the hundreds of dollars,
with the register reporting one customer found themselves within one
thousand dollars bill after a week, and I quote them,
I think it's just launch pricing adjustment. Some tasks on
new apps ran over an hour and forty five minutes
and only charged four to six dollars, but editing pre
existing apps seems to cost most overall. I spend one
K this week alone, and they told that to the register.
(07:58):
By the way, another user comp that costs skyrocket without
any concrete results, and they quote the register here. I
typically spent between one hundred dollars and two hundred and
fifty dollars a month. I blew through seventy dollars in
a night at Agent three launch, and another redditor wrote
alleging the new tool also performed some questionable actions. One
prompt brute forced its way through authentication, redoing auth and
hard resetting users password to what it wanted to perform
(08:20):
app testing on a form. The user wrote, I realized
that's a little nonsensical, but long story short, it did
a bunch of shit. It wasn't asked to. As I
previously reported, in late May early June, both open ai
and Anthropic cranked up the pricing on their enterprise customers,
leading Replet and Cursor both shifting their prices upward. This
abuse is now trickled down to the customers. Report has
(08:40):
now released an update. Unless you choose how autonomous you
want Agent three to be, which is a tacit admission
that you can't trust coding elms to build software replets.
Users are still pissed off, complaining that report is charging
them for an activity when the agent doesn't do anything,
a consistent problem I've found across redditors. While Reddit is
not the full summation of all users of every company everywhere,
it's a fairly good barometer of user sentiment and man
(09:02):
a user's piss and now here's why this is bad. Traditionally,
Silicon Valley startups have relied upon the same model, have
grow really fast and burn a bunch of money, then
turn the profit lever. AI does not have a profit
lever because the raw costs of providing access to AI
models are so high and they're only increasing that the
basic economics of how the tech industry sell software don't
(09:25):
make sense. I'll reiterate something I wrote a few weeks ago.
A large language model users infrastructural burden varies wildly between
users and use cases. While somebody asking chat gpt to
(09:46):
summarize an email might not be much of a burden,
somebody asking chat gpt to review hundreds of pages of
documents at once. A core feature of basically any twenty
dollars a month subscription could eat up to eight GPUs
at once. To be very clear, a user that pays
twenty dollars a month could run multiple queries like this
a month and there's not really a way to stop them.
Unlike most software products, any errors in producing an output
(10:08):
from a large language model have a significant opportunity cost.
When a user doesn't like an output, or the model
gets something wrong which it's guaranteed to do, or the
user realizes they forgot something, the model must make a
further generation or generations, and even with caching which anthropic
is added are told to there's a definitive cost attached
to any mistake. Large language models are for the most
(10:28):
part lacking in any definitive use cases, meaning that every
user is even with an idea of what they want
to do, experimenting with every input and output. In doing so,
they create the opportunity to burn more tokens, which in
turn creates an infrastructural burn on GPUs, which cost a
lot of money to run. The more specific the output,
the more opportunities there are of a monstrous token burn.
And I'm specifically thinking about coding with l elms. The
(10:50):
token heavy nature of generating code means that any mistakes,
suboptimal generations, or straight up errors will guarantee further token burn.
Even efforts to reduce compute cors by, for example, pushing
free users or those on cheap plans, the small or
less intensive models have dubious efficacy. As I talked about
in a previous episode, open ai split a model in
the GPT version of CHET. GPT requires vast amounts of
(11:13):
additional compute in order to route the user's request or
the appropriate model, with simpler requests going to smaller models
and more complex ones being shifted to reasoning models, and
it makes it impossible to cash part of the input.
As a result, it's not really clear whether it's saving
open ai any money, and indeed, kind I suggest it
might be costing them more. In simpler terms, it's very,
very very difficult to imagine what one user free or
(11:34):
otherwise might cost, and thus it's hard to charge them
anything on a monthly basis or tell them what a
service might actually cost them on average. And this is
a huge, huge problem with AI coding environments. But let's
talk about claud Code again. Anthropics code generate a tool.
According to the information claud code was driving nearly four
hundred million dollars in annualized revenue, roughly doubling from a
(11:56):
few weeks ago on July thirty first, twenty twenty five.
The annualized revenue works out to about thirty three million
dollars a month in revenue for a company that predicts
it will make at least four hundred and sixteen million
dollars a month by the end of the year, and
for a product that has become for a time the
most popular coding environment in the world from the second
largest and best funded AI company in the world. Is
(12:17):
that it is that fucking it is that all that's
happening here thirty three million dollars, all of which is
unprofitable after it felt, at least based on social media
chatter and discussing with multiple different engineers, that claud code
have become ubiquitous with anything to do with LLLMS and coding.
To be clear, Anthropics, so on It and Opus models
are consistently some of the most popular for programming an
(12:39):
open router, an aggregator of LM usage, and Anthropic has
been consistently named as the best at coding. Whether or
not I feel that way is irrelevant. Some bright spark
out there is going to send it. Microsoft's get hub
copilot at one point eight million paying subscribers, and guess
what that's true? In fact, I reported it. Here's another
fun fact. The Wall Street Journal report that Microsoft loses
on average twenty dollars a month per use, with some
(13:00):
users costing the company as much as eight bucks. And
that's for the most popular product. But wait, wait, wait, wait,
hold up, wait, I read some shit in the newspaper.
Aren't these LLLM code generators replacing actual human engineers? And thus,
even if they cost way more than twenty dollars one
hundred dollars or two hundred dollars a month, they're still
worth it. Right, They're replacing an entire engineer. Oh my
(13:22):
sweet summer child. If you believe the New York Times
or other outlets that simply copy and paste whatever anthropic
CEO Warrio Ama Day says, you'd think that the reason
that software engineers are having trouble finding work is because
their jobs are being replaced by AI. This grotesque, manipulative, abusive,
and offensive lie has been propagated through the entire business
and tech media without anybody sitting down and asking whether
it's true, or even getting a good understanding of what
(13:44):
it is that elms can actually do with code. Members
of the media, I am begging you stop stop doing this,
Stop publishing these fucking headlines. You're embarrassing yourself. Every asshole
is willing to give a quote saying that coding is
dead and that every execut if he is willing to
burp out some nonsense about replacing all of their engineers.
But I'm fucking begging you to either use these things
(14:05):
yourself or speak to people that do. I am not
a coder. I cannot write or read code. Nevertheless, I'm
capable of learning, and I've spoken to numerous software engineers
in the last few months, and basically I've reached a
consensus of this is kind of useful sometimes. However, one time,
a very silly man with an increasingly squeaky voice said
that I don't speak to people who use AI tools.
(14:26):
So I went and spoke to three notable experienced software
engineers and ask them to give me the straight truth
about what coding lllms can do. Now, for the purposes
of brevity, I'm going to use select quotes from what
these people said. But if you want to read the
whole thing, you can check out the newsletter first. I'm
going to read what Carl Brown of the Internet of
Bugs said, and I had him on the show a
few months back. He's fantastic. So most of the advancements
(14:48):
in programming languages, technique and craft in the last four
years have been designing safer and better ways of tying
these blocks together to create large and larger programs with
more complexity and functionality. Humans use these advancements to arrange
these blocks in logical abstraction layers so we can fit
an understanding of the lairs interconnections in our heads as
we work. Diving into blocks temporarily is needed. This is
(15:08):
where AIS fall down. The amount of context required to
hold the interconnections between these blocks quickly grows beyond the
AI's effective short term memory, in practice much smaller than
its advertised context windows size, and the AIS like the
ability to reason about the abstractions as we do. This
leads to real world code that's illogically layed, hard to understand, debug,
and maintain. Carl also said code generation AIS, from an
(15:32):
industry standpoint, are roughly the equivalent of a slightly below
average computer science graduate fresh out of school without any
real world experience, only ever having written programs to be
printed and graded. That's bad because, as he pointed out,
whereas llms can't get past this summer, in turn stage,
actual humans get better, and if we're replacing the bottom
rung of the labor market, there won't be any mid
level or senior developers later down the line. Next, I
(15:55):
asked Nick Sharesh of I will fucking pile drive you
if you mention AI again what he thought. Llms, he said,
will sometimes solve a thorny problem for me in a
few seconds, saving me some brain power. But in practice,
the effort of articulating so much of the design work
in plain English and hoping the LM emits code that
I find acceptable is frequently more work than just writing
(16:15):
the code. For most problems, the hardest part is the thinking,
and lllms don't make it that part any easier. I
also talked to Colvogi of no AI is not making
AI engineers ten X is productive. We also had in
the show recently, and he said this, llms often function
like a fresh summer intern. They're good at solving the
straightforward problems that code has learned about in school. But
they are unworldly. They do not understand how to bring
(16:37):
lots of solutions to the small, straightforward problems together into
a larger hole. They lack the experience to be wholly
trusted and trust this is the most important thing you
need to fully delegate coding tasks. In simpler terms, lms
are capable of writing code, but can't do software engineering
because software engineering is the process of understanding, maintaining and
executing code to produce functional software, and lms do not learn,
(16:58):
cannot adapt, and to paraphrase something Carl Brown said to me,
break down the more of your code and variables you
ask them to look at at once, so you can't
replace a software engineer with them. If you are printing
this in a media outlet and have heard this sentence,
you are fucking up. You really are fucking up. I'm
really neat members of the media here in this You
need to change. You need to change on this one.
(17:19):
You are doing software engineers dirty. Look, and I understand
why too. It's very easy to believe that software engineering
(17:40):
is just writing code, but the reality is that software
engineers maintain software, which includes writing and analyzing code, amongst
a vast array of different personalities and programs and problems.
Good software engineering harkens back to Brian Merchant's interviews with translators.
While some may believe the translators simply tell you what
words mean, true translation is communicating the meaning of a sentence,
which is cultural, contextual, regional, and personal and often requires
(18:03):
the exercise of creativity and novel thinking. And on top
of that, while translation is the production of words, you
can't just take code and look at it. You actually
need to know how code works and functions and wide functions.
In that way, using an LLM, you'll never know because
the LM doesn't know anything either. Now, my editor Matt
Hughes gave an example of this in his newsletter, which
(18:24):
I think i'll paraphrase. He used to live in France
and the French speaking part of Switzerland, and sometimes he
will read French translations of books to see how awkward
bits of prose are translated. Doing those awkward bits requires
a bit of creative thinking. And I quote take Harry
Potter in French, Hogwarts is boudlard, which translates into bacon lice.
Why did they go with that instead of a literal translation?
Of Hogwarts, which would be Verus Spork. I'm sorry to
(18:47):
anyone who can actually read languages, no idea, but I'd
assume it is something to do with the fact that
Poolard that Poudlard sounds a lot better than Veru Spork,
and both of them, I can say flawlessly. Someone had
to actually think about to translate that one idea. They
had to exercise creativity, which is something that an AI
in is inherently incapable of doing. Similarly, coding is not
(19:08):
just a series of texts that program as a computer,
but a series of interconnected characters that refers to other
software in other places that must also function now and
explain on some level to someone who has never ever
seen the code before why it was done in this way.
This is, by the way, while we're still yet to
get any tangible proof that AI is replacing software engineers,
because it isn't replacing software engineers, and now we need
(19:30):
to understand why this is so existentially bad for generative AI.
Of all the fields supposedly at risk from AI disruption,
coding fields or felt the most tangible, if only because
the answer to can you write code with LMS wasn't
an immediate unilater or no The media has also been
quick to suggest that AI writes software, which is true
in the same way that chat GBT writes novels. In reality,
(19:51):
lms can generate code and do somewhere some sort of
software engineering adjacent tasks, but like all large language models,
break down and go totally in saying hallucinating more and
more as the tasks get more complex, and software engineering
is extremely complex. Even software engineers who can read code
and have done so for decades will find problems they
can't solve just by looking at the code. And as
(20:12):
I pointed out earlier, software engineer is not just coding.
It involves thinking about problems, finding solutions to novel challenges,
designing stuff in a way that could be read and
maintained by others, and that's ideally scalable and secure. The
whole fucking point of an AI is that you handshit
off to it. That's what they've been selling it as.
That's why Jensen Huang told kids to stop learning to code.
(20:32):
As with AI, there's no point and it was all
a fucking lie. Generative AI can't do the job of
a software engineer, and it fails. While also costing an
abominable amount of money. Coding large language models seem like
magic at first because they, to quote a conversation with
Carl Brown, make the easy things easier, but they also
make the harder things harder. They don't even speed up engineers.
(20:53):
There's a study that showed that make them slower YEAT
coding is basically the only obvious use case for lms. Oh,
I'm sure you're gonna say, but I bet the enterprise
is doing well, and you're also very, very wrong. Microsoft,
if you've ever switched on a TV in the past
two years, has gone all in on generative AI, and
despite being arguably the biggest software company in the world
at least in terms of desktop operating systems and productivity software,
(21:16):
has made almost no traction in popularizing generative AI. It
has thousands, if not tens of thousands of salespeople and
thousands of companies that literally sell Microsoft services for a living,
and it can't sell AI. I've got a real fucking scoopyeo,
I'm so excited, and I buried it in the third
part of a four pot episode. AAH and truly twisted.
But a source that has CM materials related to Sales
(21:39):
has confirmed that as of August twenty twenty five, Microsoft
has around eight million active license so paying users of
Microsoft three sixty five Copilot, amounting to a one point
eight one percent conversion rate across four hundred and forty
million Microsoft three sixty five subscribers. Must be clear that
three sixty five is their big cash cow. This would
(22:00):
amount to if each of these users paid annually at
the full rate thirty dollars a month, to about two
point eight eight billion dollars an annual revenue for a
product category that makes thirty three billion dollars a fucking quarter.
It's productivity and business unit for Microsoft, and I must
be clear, I am one hundred percent sure these users
aren't all paying thirty dollars a month. The Information reported
a few weeks ago that Microsoft has been reducing the
(22:22):
software's price, referring to Microsoft three sixty five with more
generous discounts on the AI features. According to customers and salespeople,
heavily suggesting discounts have already been happening. Enterprise software is
traditionally sold at a discount anyway, or put a different way,
with bulk pricing for those who sign up a bunch
of users at once. In fact, I found evidence that
they've been doing this for a while, with a fifteen
percent discount on annual Microsoft three sixty five Copilot subscriptions
(22:45):
for orders of ten to three hundred seats mentioned by
an IT consultant back in late twenty twenty four, and
another that's currently running through September thirtieth, twenty twenty five,
with another Microsoft Cloud Solution Provider program. Yeah this, I've
found tons of other examples too. A Microsoft three sixty
five is the enterprise version where they sell things with
like Word and PowerPoint and sometimes teams as well. This
(23:05):
is them probably the most popular product, and by the way,
they even manipulate the numbers a little bit there. An
active user is someone who has taken one action on
any Microsoft three sixty five app with Copilot in the
space of twenty eight days, not thirty twenty eight. That's
so generous, now, I know, I know that word active.
Maybe you're thinking ed, this is like the gym model.
(23:26):
There are unpaid licenses that Microsoft is getting paid for. Fine, fine, fine,
fucking fine. Let's assume that Microsoft also has based on
research that suggests this can be the case for some
software companies another fifty percent four million paying Copilot licenses
that aren't being used. That's still twelve million users, which
is around two point seven percent conversion rate. That's piss,
(23:48):
poor buddy, that's piss, Paul, that's pissy. It sucks. It's bad, Doodoo.
Well I just said pp I guess anyway, very serious,
very serious podcast. But why aren't people paying for Copilot? Well,
let's hear from someone who talked to the information and
I quote, it's easy for an employee to say, yes,
this will help me, but hard to quantify how. And
if they can't quantify how it will help them, it's
not going to be a long discussion over whether the
(24:10):
software is worth paying for. Is that good? Is that good?
Is that what you want to hear? It isn't. It isn't.
That's that's the secret. It's not. It's bad. It's really bad.
It's all very bad. And Microsoft through sixty five Copilot
has been such a disaster that Microsoft will now integrate
Anthropics models to try and make them better. Oh one
other thing too. Sources also confirm GPU utilization, So how
(24:34):
much the GPUs set aside for Microsoft through sixty five? Yeah,
their enterprise codpile. It's barely scratching the sixty percents. I'm
also hearing the share Point, which is an app they
have with over two hundred and fifty million users, has
less than three hundred thousand weekly active users of their
copilot features, suggesting that people just don't want to fucking
use this. Those numbers that from August, by the way,
(24:57):
and it's pathetic, and it must be clear. If Microsoft's
doing this badly, I don't know how anyone else is
doing well, and they're not. They're all failing. It's pathetic.
But I've spent a lot of time today talking about
AI coding, because this was supposed to be the saving grace,
the thing that actually turned this from a bubble into
an actual money minting industry that changes the world. And
I wanted to bring up Microsoft through sixty five because
(25:18):
that's the place where Microsoft should be making the most money.
It's the most ubiquitous software, it's their most well known software,
and they're not eight million people eight million people. I've
run that by a few people and everyone's made the
same Oh God noise. It's quite weird, the old God
noise and the numbers. But this just isn't happening. Things
are going badly and it really only gets worse from here,
(25:41):
and I'm going to tell you more tomorrow in the
final part of our four part Thank you for your
patience and thank you for your time.
Speaker 2 (25:55):
Thank you for listening to Better Offline. The editor and
composer of the Better Offline theme song is Matasowski. You
can check out more of his music and audio projects
at Matasowski dot com, M A T T O S
O W s ki dot com. You can email me
at easy at Better Offline dot com or visit Better
Offline dot com to find more podcast links and of
(26:16):
course my newsletter. I also really recommend you go to
chat dot Where's youreaed dot at to visit the discord,
and go to our slash Better Offline to check out
our reddit. Thank you so much for listening.
Speaker 1 (26:28):
Better Offline is a production of cool Zone Media. For
more from cool Zone Media, visit our website cool Zonemedia
dot com, or check us out on the iHeartRadio app,
Apple Podcasts, or wherever you get your podcasts.