August 1, 2024 • 36 mins
Welcome back to the Generative AI Meetup Podcast! After a brief hiatus, hosts Mark and Shashank return with fresh insights from a recent event held in a historic Silicon Valley location. This episode delves into the role of AI agents in transforming workflows and enhancing various business processes. The discussion kicks off with highlights from the meetup, attended by over 100 enthusiasts and professionals, and hosted at the birthplace of the integrated circuit. The conversation then shifts to the latest in AI news, focusing on Facebook's open-source Lama 3.1 model, which is setting new standards in AI performance and accessibility. Tune in to learn how these advancements in AI are not only pushing technological boundaries but also democratizing AI capabilities for startups and developers around the globe.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
All right, well, hello everybody and welcome to another episode of the Generative AI Meetup
(00:09):
Podcast.
So, I think we missed it last week, but don't worry, we're back this week better and stronger
than ever.
The reason we weren't around last week is because we were doing what we do best.
We ran a meetup.
It was a fantastic event, I think.
We had about, what do you think we had, Shashank?
(00:30):
Maybe about 100 people that showed up.
That was 100?
Yeah, it was fantastic.
So, we did a little event on AI agents.
We had people from all over the world come to the event, which was quite cool.
And it was in one of the most historic buildings in Silicon Valley.
So, it was in the building where they had, I think, this is a fair child semiconductor,
(00:55):
which was one of the very first companies that kind of started Silicon Valley.
I think it was the first company that made the integrated circuit, which is kind of
important, you would think, the integrated circuit.
I think it was the first commercially available integrated circuit developed at Fairchild
semiconductor.
Yeah, I mean, that's pretty cool, right?
(01:16):
So, we were in that building.
So thank you so much to Apex AI for opening up their office base to us, shouted at the
Apex AI, not sponsored, but they helped us out.
So we'll give them a little shout out on the podcast.
Gracious hosts, wonderful hosts.
So they are doing some self-driving car stuff, and they're working with a lot of the big
(01:41):
name auto brands.
So I think that you may not know them, but if you're going to be driving some cars, I think
they were working with like Volkswagen and whatnot.
So I think that we will see, you will probably see the brains of Apex AI real soon.
(02:02):
So that's kind of cool.
So anyways, for those that don't know us, I'm Mark, and with my co-host, Shashank, we run
Meetups in the Bay Area, and we talk to a lot of really interesting and cool people every
single week about Generative AI.
And we just want to kind of share with you some of the ideas and topics that we discuss
(02:27):
every week.
So Shashank, what's in the news this week?
There's been a lot of different stuff in the news.
No big announcements.
I think the biggest thing has been Facebook's Lama 3.1 model.
I think they announced it last week, but we weren't here.
So maybe we can touch upon that a little bit.
(02:49):
But to give people a high level overview, I think a lot of these models are converging
towards the GPT-4Os performance level, everything from Gemini to Claude, Lama 3.
It seems like we're reaching a plateau in these models' capabilities, and they're all reaching
(03:13):
multimodal.
They can understand video.
They can understand text.
I don't know about audio.
Maybe audio.
I think GPT-4O can understand audio.
And definitely text across a variety of domains like mathematics, common sense reasoning,
medicine, legal, etc.
(03:34):
So it seems like OpenAI doesn't have a stronghold on this market anymore.
And I love that Facebook is open sourcing this whole thing too.
Do you know if they open sourced the biggest model?
What is it for?
The 4 or 5B1?
Yeah.
I think so.
I think they have.
(03:55):
I'm pretty sure that you can just run it on your own hardware.
Oh, that's crazy.
Although, I don't know who has hardware that could run that.
I mean, I don't have a couple of H100s line around.
That is a lot.
Yeah.
Well, I mean, if you're a startup, maybe.
Yeah.
I mean, if you're a startup, you might be able to have one.
I guess.
But there's a lot of money.
(04:17):
And I also wonder if you really need something that big because it's like, do you really need
that high quality of a model for the majority of your tasks?
Probably not for the majority of tasks, but for a few key things, I think it's always better
to have the best model.
I mean, when I go and try to find a model to use, I'm just paying the best GPT-40 model.
(04:38):
I'm not trying to pull the local sub 10 billion parameter model.
Yeah.
I mean, sure.
I think that, in general, if you can, you might as well run the best.
But I think that it's not always better to run the largest model, right?
Sure.
(04:59):
Class or cost, time, infrastructure to host this kind of model, right?
Memory, CPU, GP resources.
But I do think that these biggest models are only available to the largest of companies
with like the $10 plus billion valuation and Meta, Google, Amazon, et cetera.
(05:21):
But now that these big models are being open-source, I think these smaller, YC companies, maybe
not the early stage, but Series A, B. They can try to take these open-source models and
build cool applications out of it, maybe focus on a specific domain, legal, medicine,
et cetera.
That's true.
That's true.
Although I would think that if you're going to focus on a specific domain, you could probably
(05:42):
start training or fine-tuning some of these smaller ones, like maybe the Lama 70B or
even 8B, in a lot of cases, is I think probably good enough depending on your use case.
I mean, sure.
Like if you want the latest and greatest, if you have some really complicated use case,
I think like, yeah, use the best, use the Lama 405B or use GPT4, the Anthropics, what
(06:06):
is it, the Clawed 3.5, or is it the Clawed Son at 3.5?
Like you know, use the best one.
But I think like, you know, those models, as you mentioned, they're expensive.
They require a lot of energy and hardware to run.
I think that like, oftentimes you can get like a good enough response with something, you
(06:27):
know, far cheaper.
And also, you have to, if you have like one of the smaller models, you don't have to deal
with any latency because you could even potentially run it like on the edge or like on someone's
device as well.
So, like, there's also the privacy aspect of it.
So I think like at this point, like, it's really about like the trade-offs because I mean,
(06:48):
it's really, I think some of these smaller ones are just good enough for like a lot of use
cases really.
Because I mean, like, if you're going to like be doing something like, for example, like
maybe like content moderation, I think it could be like a use case, right?
So like, if I'm like monitoring like my chat room, right?
I want to look for, let's say hate speech because they like, hey, like is this hate speech
(07:10):
or not hate speech?
Or like, you know, some inflammatory content.
You could just have, I think, a smaller model be able to identify that just fine.
I don't need like, you know, a Lamborghini, if I'm just like going back and forth to the
supermarket, you know?
You don't want to take a shotgun to try to remove the pests in your house.
Right.
(07:31):
I mean, like, it works, which is a little much, right?
A little overkill.
Yeah.
I can see very specific use cases, let's say customer service or something where you
just want an FAQ bot that gets people's order numbers or answers questions about their
shipping time for a drop shipping, Shopify marketplace, then you don't really need a sonnet or a
(07:53):
TFT 40 for those kinds of tests.
Yeah, exactly.
So I mean, like, for like, if I'm just like playing around, I want to use the latest and
greatest and know like what's possible.
But I think like, once you like kind of niche down onto your particular task, I mean,
like the smaller ones five.
I mean, I think that most of the cases just completely fine.
(08:14):
For the startup ecosystem at large, this is still fantastic news.
Yeah.
Yeah.
And you mentioned that that's kind of, I think Mark Zuckerberg is part of his strategy.
He wants people to start using llama just for all the things.
That's right.
He was talking about this again yesterday or a couple days ago.
He was at Nvidia's SIG graph conference.
(08:38):
And he was sitting with the Jensen Huang, the CEO of Nvidia and talking about some of their
high level strategies for tackling this problem.
And they were talking about how access to GPUs has always been a problem.
And Zuck was giving Jensen a little bit of, you know, a hard time for not delivering GPUs
(08:59):
on time and being a supply constrained.
Then talking about how open sourcing has worked really well for Nvidia in the past.
How a lot of the actually Nvidia or for one of them, how they open source some of the networking
switches and that became an industry standard and started being used by everyone else.
(09:22):
And then they were able to take all the improvements that the open source communities made to these
networking switches and then incorporated back into their architecture.
So they're planning to do something similar with llama as more and more people find bugs
and vulnerabilities in llama as they start using it themselves and make improvements.
They are able to, you know, absorb these improvements from the open source community
(09:45):
and then make the next version of llama bigger, better, faster, more secure.
That's true.
It really feels like some sort of virtuous flywheel for like Facebook where they're going to create
new bigger models.
Everybody's going to go.
So I think I remember seeing like the day after llama through poem was launched.
I saw like a couple hackathons just being posted like, oh, emergency hackathon.
(10:09):
We need to like build stuff on top of this llama 3.1.
It's exciting.
And now like if people start building stuff on llama 3.1, then like people are going to,
you know, kind of just be more used to like the Facebook stack.
And then they're going to become like the standard or because we build on top of it.
And then like then Facebook is going to get a bunch of stuff for free.
(10:30):
They're going to get the majority of the mindset or the developers using it.
And also like people are going to use them because they can run it like on prem.
So Facebook is going to get a bunch of free development work.
And then like I mean, Facebook isn't necessarily making money off these LLMs.
They're making money off of like showing ads.
So like if people build a bunch of cool stuff, then Facebook could probably show better ads
(10:54):
their users and then you know, make more money.
Facebook's docked to the moon.
Yeah.
And to make a small distinction, they're not open sourcing the product.
They're open sourcing the underlying technology behind the product.
So the product that they've integrated into WhatsApp and Facebook Messenger and a bunch
of other products is using llama 3 and lots of other rag type filters to pull context
(11:21):
from their social networks and your personal chat history, etc.
So they're adding a lot of things on top of llama 3 that they're going to keep close
source and maintain their competitive advantage.
Otherwise like they give everything out for free that'll kind of make them lose their
advantage for sure, for sure.
(11:42):
But you know, it really feels to me like it's becoming a race to the bottom and nobody is
going to be making money by selling these LLMs.
They're selling LLMs.
They're selling LLMs.
They've got a couple foundation models.
I don't know how competitive they are.
Oh, how good they are.
But everyone is selling foundation models.
(12:05):
But like they're actually selling the foundation model or they're giving away for free.
They might be giving it away for free.
Yeah.
But like the thing is, if you think about it, there's actually very few companies right now
that are making money directly from LLM, like LLM as the product.
I mean, the only ones that I can think of are on Thropic and Chatchapetit.
Google.
(12:26):
Oh, Google, right.
True.
Pi, character AI, but isn't Pi, Amazon has one for enterprise.
Yeah, but like I think meta AI, you can pay for it.
X AI, you can pay for it.
I actually meta, you can't pay for it.
Meta is free.
X AI, you can start paying for it.
Oh.
Oh.
(12:46):
Whoa.
Okay.
Anyways, let's get with that lesson.
All right.
Yeah.
Let's just ignore that.
So I guess you're right.
I guess there's like a few, but I don't know.
Like it seems to me that like all these companies are going to have a harder and harder time
justifying like $10 a month or $20 a month or whatever it is.
Yeah, you're right.
Like Google has like the Gemini Advance, Chatchapetit has their GPD 4 for sale.
(13:12):
Pi, is that free?
Oh, that was free.
It is free, but I do think they have a paid version too.
Okay.
So anyways, like the fact that all, a lot of these are going to be open source and people
can just run on their own.
I think the only reason why people may pay is just because they don't want to have to
(13:33):
host the model themselves and then want to use the latest and greatest.
But the fact that a lot of these smaller models are kind of good enough, as I mentioned
before, right?
Like a lot of people won't feel the need to pay like Facebook to run it because if they
have like a specific use case that they want to use the model for, they might just run
(13:57):
on their own hardware.
Well, that's why these companies need to do better and better with every release to make
it worth paying the subscription fee.
Otherwise, people are just going to turn to these cheaper models.
So OpenAI definitely keeps pushing the frontier.
They released Search GPT.
Should we talk about that?
(14:18):
We can.
I'm on the wait list now.
Yeah.
So, this is supposed to be the competitor to Google search.
And especially the Gemini summaries that you see at the top of Google search, if you have
that enabled or perplexity, which was dubbed the Google Killer, which uses these LLMs.
It's actually LLMs look agnostic.
(14:39):
They use whatever LLMs that you want.
You can be from Anthropic, ChatGPT, Google, and they use these LLMs to search the web, filter
the results, and summarize these search results.
And multiple steps if it needs to.
So it's cool that ChatGPT is doing this finally.
I feel like it was about time.
Because Search has been like the first place that people go to for a long time and to integrate
(15:02):
that with your LLMs.
That's fantastic.
It is a little interesting that Bing has been funding OpenAI for a long time and they
wanted to try to take over the search market with ChatGPT.
But now if ChatGPT has searched built in, are people really going to go to Bing?
(15:26):
I don't know.
Yeah.
And I also wonder if this is going to hurt Google.
Because if Google started to go down after that and find the set or something.
The thing is if people start just getting the answer from ChatGPT and they go to that
over Google, Google doesn't in my opinion have a super big moat.
(15:47):
I mean they have a big moat, but like it's not that big.
Well the people who work on Google search are some of the smartest people that I know and
we have the most technologically advanced stack to do any kind of search.
The latency that we have is insane.
I don't think ChatGPT can match like anywhere close to the latency that we get for both the
(16:11):
Google search and the Gemini results on top.
It's blazing fast.
That's true.
It is very fast.
But I don't know.
I think the thing is, at least for me I feel like a lot of times my Google search results aren't
exactly what I want.
Something I have to click through a lot of links.
(16:33):
I think there's a lot of SEO content.
I actually personally don't use Google search anymore.
You are one of the few outliers that you see to make that clear.
I mean that's true.
I am an outlier.
I don't use Google search at all.
I use a different search end called Kaggy.
So Kaggy like K, A, G, I, it's a paid search engine.
(16:55):
I paid ten bucks a month for it.
It's not sponsored.
Not sponsored.
I pay them money.
If they want to give me money to talk about them, that'd be fantastic.
Although they're a pretty small company.
But yeah, I think I personally prefer the results over a little bit.
Maybe I'm biased.
(17:19):
And actually Kaggy has a similar instant answer like Google search as well.
So it's got all the same things.
It should be cool.
They're probably using a smaller model.
To get that kind of latency on a bigger model would be hard.
But I feel like you're paying for it.
Maybe that subscription fee goes towards data centers.
(17:40):
Probably.
Yeah.
But I do feel like sometimes Google is a little bit slow for the instant answers.
So like it does buffer.
It doesn't render immediately.
Yeah.
So like for example, I was looking at some Android developer in my full time and my full
time job.
And I was looking through the Android developer documentation.
(18:02):
I wanted to look up some, you know, how to do some things.
And I saw that like Google started adding instant answers to their Android developer documentation.
And it was too slow to be useful.
So it also gave me the exact same result as what they just had lower in the page.
So it didn't really help.
(18:22):
But that's helpful.
I mean, I guess I don't have to scroll down the page or sift to all that content, which
is fine.
But I also use like command app and like find it immediately and then get like the actual
documentation and not like some summary of it.
So I don't know.
I mean, like, I think like the concept is cool, but I haven't found it like super useful yet
(18:46):
in my workflow.
Yeah, I don't know.
But I do think that the search, what do they call search GPT or I heard it's GPT.
Yeah.
It's exciting.
It's exciting and there's more competition out there.
So I welcome the competition and I know like I think Google's like still the top dog, but
(19:11):
there I think still is like room for somebody else to come in and definitely disrupt them.
Definitely.
This is a new world.
And I have been waiting for opening it to do this for a while.
I'm glad they finally did it.
And looking forward to it.
Yeah.
Yeah.
It's cool stuff.
So let's see what else is in the news.
(19:33):
You mentioned that, oh, actually here's something small, but pretty interesting.
Stability AI, they introduced a new thing called stable fast 3D.
Kind of cool.
So what it is, you can give it an image and it will generate a 3D model and it'll do it
(19:54):
really fast.
So I think you could probably use it.
I would imagine like the main thing would be like maybe video games.
So like you could take an image of like, I don't know, like a couch or like a cup or whatever
and then it'll go in and then like make a 3D model of that thing.
So where you used to have to you know, pay some sort of like graphic designer, I don't
(20:16):
know what you call like a 3D artist.
I don't know what like the name of that is digital artists, digital artists.
So anyways, you had to pay one of those people to you know, spend all day making your
different 3D models for the video game.
Now you could just like potentially, I mean honestly, you could probably do like a whole
end to end thing where maybe you have like an AI model.
So I could say like, hey, chat to the tea, create me a picture of a, I don't know, like
(20:40):
a car and then I could feed that car into this model and then now I have like a cool new
car for my video game.
So I think that we're going to start seeing like really good video games in the near future.
That is really cool.
I'm looking at their website for some more information and it's really cool how they take
(21:01):
the image and construct a 3D object with information about the what is behind the object.
Because oftentimes we get a lot of artifacts and like missing, um, depth and things like that.
But I think it has enough understanding of what you know, different kinds of common objects
are like a teddy bear or a sandwich or a couch where it can fill in the back in like a reasonable
(21:26):
way.
And they show us a show of video for like 4D outputs, which is like a 3D model with some
kind of animation mostly for characters.
So if it's like a dolphin, it'll be like flipping its tail or if it's like a camel, it'll
be walking, moving its limbs and head and it looks really cool.
Yeah, and also it seems like it's open weight.
(21:50):
That is surprising because stability has been going through a lot of turnover in their
leadership.
I think I'm odd, their founder, CEO left and they've been trying to figure out how to convert
from this whole open, free, everything approach to trying to make money and staying afloat is
very expensive to build and train all these models.
(22:13):
They've gone from image which they started with to language, which they added on later,
to video, now 3D.
That must be very, very expensive.
Yeah, I'm really curious like how a lot of these companies are going to stay in business.
Well, stability was just bankrupt, but this one guy who made a huge fortune and he's just
(22:35):
like, you know what, I want to give this to the world to kickstart the new generation of
hackers and independent developers to give them, you know, democratize this technology and
not keep it in the hands of open AI.
I mean, that's fantastic or mid-journey, which is the only competitive image generation
tool of the time.
(22:55):
Yeah, I mean, I think that's like great for the world, I fully support it.
The only thing is is like, I wonder if that's sustainable because I mean, clearly it's
not for them because the thing is is like, it seems like right now the only way you can
get these high quality models is by just throwing more compute at the problem, like more energy,
(23:16):
more GPUs, more data, just more.
And it seems like to scale.
And without any kind of like algorithmic advancements, it's just going to be only the biggest companies
in the world that are going to be able to train these models.
And I think it's even getting really expensive for the biggest companies.
I mean, think about like the biggest companies right now.
I mean, like, what's number one?
I think it's like, it's kind of sheer solid time like apples up there and video.
(23:41):
Video is close.
Yeah, actually not this week.
They've taken quite a hit.
Like Google's up there, Amazon, you know, whatever, like all these big companies.
If we're talking about individuals, I think Elon's up there too.
If we're talking about like entities who have a lot of money to throw the problem.
Yeah.
So his new X AI venture, which is trying to, you know, come up with a new LLM and multi-modal,
(24:06):
probably model.
He is trying to build a data center in Memphis and get access to close to 150 megawatts of energy.
And we were talking about this earlier for context.
I think your average home, you know, maybe like a three bedroom home would be in the low,
(24:28):
maybe 10 digit, sorry, it's like 10 kilowatts.
10 kilowatts, yeah.
Not 10s of kilowatts.
10 to be like a big home, I think.
10 kilowatts.
I think it's like two kilowatts.
What I thought, yeah, two, like a small studio one bedroom or something.
Yeah.
And then like 10 to be like a higher-sager, just yeah.
And so he's raising like a hundred thousand times or 10,000 times, he's looking for 10,000
(24:53):
times more energy than what a single family home would be using.
Seems big.
Do you know any reason why he chose Memphis?
No idea.
Yeah.
Interesting.
I mean, like there's nothing wrong with Memphis.
Big fan of Memphis, great place.
I'm just curious because like, you know, all of his companies are in like California or Texas.
(25:15):
So like, Memphis seems interesting.
He had a big gigafactory in Nevada too.
I think the way he approaches these problems is goes to the person who can offer the most
value or instead of person, it would be government entity state.
Maybe there's access to cheap energy.
(25:35):
Maybe there's big subsidies.
So for this particular decision, it seems like Tennessee offered ample access to power
and the ability to build quickly.
Maybe removing some of the red tape.
That seems good.
Yeah.
I mean, I guess it doesn't really matter where you build it because like all that really matters
(25:58):
is like, I think the main thing would be like you want an area for going to build a data
center.
You probably want it to have like low natural, low rates of natural disaster, political
stability, political stability, public benefit, water to cool all these data centers.
Yeah.
Cheap energy.
(26:19):
I don't know, just some, but it doesn't need to be like in Manhattan or something like that.
Yeah, because you're training these models.
You're not making them available.
You can make them available in other data centers close to metros.
Right.
Exactly.
I mean, like Memphis, I mean, that's still like a decently sized metro that is close to
(26:41):
a lot of people.
So that's good.
Maybe.
I mean, yeah, there's a lot of people living in Memphis.
It's a good place.
I mean, you want to cover the West Coast and East Coast though if you're talking about the
US.
Well, I mean, maybe Chicago.
You're just training it.
You don't need to make it available.
Yeah.
So I mean, like it could be anywhere.
You can do this in Alaska.
(27:01):
It doesn't matter.
Yeah, that's fair.
Like, it's fine.
I could go and make it.
It doesn't matter.
You could.
I mean, just all you need is like a nice little data cable that goes from Antarctica to,
like, I don't know, the closest city, like somewhere like in Alaska.
You don't need cables.
He's got Starlink.
Beam it all into the atmosphere.
Oh, yay.
Oh, actually, I was thinking, I don't know what the throughput of that would be though.
(27:25):
Good enough.
I don't know.
I mean, like, the model aren't that big.
A lot of data though.
That's true.
That's true.
How would you get the data to Antarctica?
Just a container ship full of hard drives.
You could actually, you should never underestimate the power of, like, semi-trucks, like, filled
(27:47):
with hard drives, like, barreling down the highway.
I'm sure they're using hard drives to back up some of these models.
I mean, why wouldn't you?
I mean, like, we're also going to put it tapes.
But tapes are kind of slow.
They are racing.
Yeah.
Archival storage.
Yeah.
I mean, like, you could, I guess.
They do.
(28:07):
Yeah.
If I was going to do it, I would just probably use, like, hard drives.
I mean, because, like, the models aren't like, I don't actually know how much data they
use.
I imagine it's a lot.
But I think the thing is, is, like, I think it's not the data that's necessarily the bottleneck.
It's more like the compute.
Because, like, you need to, like, process the data, like, many times, and, like, lots of different
(28:33):
ways.
So, it's like, I think that, like, probably, like, honestly, just, like, a few, like, hard drives,
like, oh, maybe, like, a petabyte of data.
I mean, that's not, like, how many hard drives do you need for a petabyte data?
Because if you have, like, petabytes, like, a thousand terabytes, right?
So, I mean, you get, like, a one terabyte, I think they're, like, ten terabytes drives now.
(28:54):
So, I mean, what do you need?
Like, a hundred of those?
I mean, it's not that much space.
I mean, I could probably fit that in, like, let's read them.
The back of, like, you know, like a standard car.
It's not like, you guys are crazy, right?
Right.
Data isn't the, like, bottom, like, it is energy.
I don't think Elon's going to get all this energy immediately.
There have been a couple articles about how a lot of companies are freezing energy shortages
(29:18):
to train these models.
It's getting more expensive.
And it's like, especially Facebook, the way they have improved the performance for some
of their models is just by training it longer.
So, I think maybe five years ago or something, they thought that there was a limit to how
good these models can get.
(29:38):
And it plateaus, which is kind of true.
It does appear to plateau, but it doesn't actually plateau.
It just improves much slower, the longer you train it.
So that's what Facebook has been doing.
They've just been continually training it for longer and longer, even though it doesn't
improve as much, but it does improve, like, very miscule amounts.
So basically, you're saying, just throwing money at the problem and then just making it
(30:03):
about as good as you can get it.
Keep making it better.
Keep making it better.
Keep throwing more money at it.
Yeah.
I feel like, is that sustainable?
Are we going to start to have nation states be the only types of people who can train these
models?
I mean, at a certain point, it's going to be none of the big companies are going to be
able to afford it.
And the only thing you're going to have is the US government, the Chinese government,
(30:27):
it's like people who are lobby should be allocated money to healthcare or training the next model.
It might be.
I mean, you could like legitimately see that future where it's only like the few largest
governments on earth.
Because I mean, the thing is, there's like a lot of these big mega companies.
I think they're being stretched.
(30:48):
And there's nobody who has more money than that.
I mean, like what country has more money than like Facebook?
It's like this very few.
Obviously the one that we're in right now.
Well, I mean, like the United States, sure, like China, Japan, Germany, Germany, right?
(31:10):
But like, I mean, once you get past the top 10, I mean, there's not many countries that
have like more money than those companies.
Sure.
So these companies are massive, yeah.
Right?
I mean, these companies are like so rich and valuable that like no one will be able to do
it.
And then it's just going to become like a government thing like a like who knows is like maybe
(31:34):
we'll start seeing these models are going to be like water or it's like it's just going
to not be like PG&E where you get your water or like your electricity.
It'll just be like you're going to pay like your monthly fee and get your model.
It's like you know, regulated by the government.
It'll just be like a pseudo government entity kind of like honestly, be like the internet
(31:56):
almost like I feel like the internet is almost like an equivalent to telecom or railroads
or some fundamental infrastructure.
Right.
Right.
It'll be fun about the infrastructure.
It's an outer electricity.
Yeah.
I mean, I don't see why it wouldn't be.
It feels like it's going that way.
But I mean, at least in the US, a lot of these things are privatized.
(32:18):
PG&E is a private.
Yeah, it's like private technically.
Yeah, private it.
And I mean like same with like the telecom companies like I mean like what are the big
ones in the US like 18T for rise T mobile.
I think it's consolidated into those two.
Yeah.
Three right.
(32:38):
T mobile as well.
But separate.
Yeah.
But I mean, the thing is is like they're not like the government.
But like they're highly regulated.
Highly regulated.
Yeah.
Same with PG&E like I know that like technically private company, but it's like highly regulated.
Like I mean, the government has essentially like given them monopolies and like I would
(33:02):
not be surprised if even though like open AI is like a private company that's like pseudo
owned by Microsoft right now.
Like I would not be surprised if they just start being like highly regulated by the government
and it becomes like almost like another R of the government.
(33:25):
Like maybe we'll start seem like the biggest models being controlled by like I don't know
like the US military or something.
I could see it being similar to Uber.
How they faced a lot of regulation.
They're still a very private company, but they do have a lot of government oversight.
And each country stayed that they operate in has its own set of rules and regulations that
(33:45):
it has to comply with.
Yeah.
I mean, but the thing is I think the difference with Uber is like there isn't any like fundamental
reason why people can't compete with them.
Like it's not like they process so much data.
I mean, they're like a big network, right?
They're like so regulated, but I think that like they're regulated in order of magnitude
less than like the utility company.
(34:07):
Sure.
So that's why I would sort of think that like like it's a big if, but if these model like
if we don't see like major algorithmic changes and if these models, you know, let's say like
in five years from now, it takes like a trillion dollars to train them, but in these new models.
(34:29):
If like if it just happens that the only way to make these models better is by just throwing
more data at the problem, more energy, more money.
But only entities are going to be able to do it are going to be governments.
And then like at that point, maybe it was give it away to like the masses and like you just
pay for it like a utility bill.
I don't know.
I can see it happen.
Yeah, interesting problem because like now that all these models are being given away
(34:53):
for free, how do they make money?
It's like apart from their main business, you know, ads for Google and ads for Facebook,
okay, there's to figure out how to make money from these things. And especially if video generation
takes off, we still don't have like widespread video generation tools available.
It's been advertised.
I think Google also announced it's new Vio model or DeepMind did and I'm on the wait list.
(35:20):
I just signed up.
But once that takes off video generation training these models is going to be very computationally
intensive.
Yeah, and also just like as people use these models more for just like regular things, right?
Like once we see like more agent work flows, once we see more people just like integrate
(35:41):
them into apps, just they get more popular.
That's just going to like really increase the need of computing.
I think maybe at some point we'll just start needing to use like a license for you to
like the energy that we need.
Yeah.
So we're still along with a long ways away from there.
Not that far away from the distance.
You never know.
(36:02):
Maybe the exponential growth is closer than we think.
It is closer than you think.
I mean, yeah, it's like as they say, if you have like a like bacteria in a lake that's
doubling every, you know, every day, right?
You can't handle that many double links, right?
(36:22):
Double links is a dangerous dangerous phenomenon.
And I mean, like the computing power is doubling every two years.
Yeah.
So I mean, we're going to need it soon enough.
Dyson sphere.
All right.
Let's start planning for the Dyson sphere then.
All right.
Sounds good.
I know we're just about out of time.
But anyways, all right guys, if you have any like ideas on how we can avoid the Dyson
(36:45):
sphere and just like make the models better, let us know.
We'll work on it.
So we'll see you in the next one until next time.
All right, well, hello everybody and welcome to another episode of the Generative AI Meetup
(00:09):
Podcast.
So, I think we missed it last week, but don't worry, we're back this week better and stronger
than ever.
The reason we weren't around last week is because we were doing what we do best.
We ran a meetup.
It was a fantastic event, I think.
We had about, what do you think we had, Shashank?
(00:30):
Maybe about 100 people that showed up.
That was 100?
Yeah, it was fantastic.
So, we did a little event on AI agents.
We had people from all over the world come to the event, which was quite cool.
And it was in one of the most historic buildings in Silicon Valley.
So, it was in the building where they had, I think, this is a fair child semiconductor,
(00:55):
which was one of the very first companies that kind of started Silicon Valley.
I think it was the first company that made the integrated circuit, which is kind of
important, you would think, the integrated circuit.
I think it was the first commercially available integrated circuit developed at Fairchild
semiconductor.
Yeah, I mean, that's pretty cool, right?
(01:16):
So, we were in that building.
So thank you so much to Apex AI for opening up their office base to us, shouted at the
Apex AI, not sponsored, but they helped us out.
So we'll give them a little shout out on the podcast.
Gracious hosts, wonderful hosts.
So they are doing some self-driving car stuff, and they're working with a lot of the big
(01:41):
name auto brands.
So I think that you may not know them, but if you're going to be driving some cars, I think
they were working with like Volkswagen and whatnot.
So I think that we will see, you will probably see the brains of Apex AI real soon.
(02:02):
So that's kind of cool.
So anyways, for those that don't know us, I'm Mark, and with my co-host, Shashank, we run
Meetups in the Bay Area, and we talk to a lot of really interesting and cool people every
single week about Generative AI.
And we just want to kind of share with you some of the ideas and topics that we discuss
(02:27):
every week.
So Shashank, what's in the news this week?
There's been a lot of different stuff in the news.
No big announcements.
I think the biggest thing has been Facebook's Lama 3.1 model.
I think they announced it last week, but we weren't here.
So maybe we can touch upon that a little bit.
(02:49):
But to give people a high level overview, I think a lot of these models are converging
towards the GPT-4Os performance level, everything from Gemini to Claude, Lama 3.
It seems like we're reaching a plateau in these models' capabilities, and they're all reaching
(03:13):
multimodal.
They can understand video.
They can understand text.
I don't know about audio.
Maybe audio.
I think GPT-4O can understand audio.
And definitely text across a variety of domains like mathematics, common sense reasoning,
medicine, legal, etc.
(03:34):
So it seems like OpenAI doesn't have a stronghold on this market anymore.
And I love that Facebook is open sourcing this whole thing too.
Do you know if they open sourced the biggest model?
What is it for?
The 4 or 5B1?
Yeah.
I think so.
I think they have.
(03:55):
I'm pretty sure that you can just run it on your own hardware.
Oh, that's crazy.
Although, I don't know who has hardware that could run that.
I mean, I don't have a couple of H100s line around.
That is a lot.
Yeah.
Well, I mean, if you're a startup, maybe.
Yeah.
I mean, if you're a startup, you might be able to have one.
I guess.
But there's a lot of money.
(04:17):
And I also wonder if you really need something that big because it's like, do you really need
that high quality of a model for the majority of your tasks?
Probably not for the majority of tasks, but for a few key things, I think it's always better
to have the best model.
I mean, when I go and try to find a model to use, I'm just paying the best GPT-40 model.
(04:38):
I'm not trying to pull the local sub 10 billion parameter model.
Yeah.
I mean, sure.
I think that, in general, if you can, you might as well run the best.
But I think that it's not always better to run the largest model, right?
Sure.
(04:59):
Class or cost, time, infrastructure to host this kind of model, right?
Memory, CPU, GP resources.
But I do think that these biggest models are only available to the largest of companies
with like the $10 plus billion valuation and Meta, Google, Amazon, et cetera.
(05:21):
But now that these big models are being open-source, I think these smaller, YC companies, maybe
not the early stage, but Series A, B. They can try to take these open-source models and
build cool applications out of it, maybe focus on a specific domain, legal, medicine,
et cetera.
That's true.
That's true.
Although I would think that if you're going to focus on a specific domain, you could probably
(05:42):
start training or fine-tuning some of these smaller ones, like maybe the Lama 70B or
even 8B, in a lot of cases, is I think probably good enough depending on your use case.
I mean, sure.
Like if you want the latest and greatest, if you have some really complicated use case,
I think like, yeah, use the best, use the Lama 405B or use GPT4, the Anthropics, what
(06:06):
is it, the Clawed 3.5, or is it the Clawed Son at 3.5?
Like you know, use the best one.
But I think like, you know, those models, as you mentioned, they're expensive.
They require a lot of energy and hardware to run.
I think that like, oftentimes you can get like a good enough response with something, you
(06:27):
know, far cheaper.
And also, you have to, if you have like one of the smaller models, you don't have to deal
with any latency because you could even potentially run it like on the edge or like on someone's
device as well.
So, like, there's also the privacy aspect of it.
So I think like at this point, like, it's really about like the trade-offs because I mean,
(06:48):
it's really, I think some of these smaller ones are just good enough for like a lot of use
cases really.
Because I mean, like, if you're going to like be doing something like, for example, like
maybe like content moderation, I think it could be like a use case, right?
So like, if I'm like monitoring like my chat room, right?
I want to look for, let's say hate speech because they like, hey, like is this hate speech
(07:10):
or not hate speech?
Or like, you know, some inflammatory content.
You could just have, I think, a smaller model be able to identify that just fine.
I don't need like, you know, a Lamborghini, if I'm just like going back and forth to the
supermarket, you know?
You don't want to take a shotgun to try to remove the pests in your house.
Right.
(07:31):
I mean, like, it works, which is a little much, right?
A little overkill.
Yeah.
I can see very specific use cases, let's say customer service or something where you
just want an FAQ bot that gets people's order numbers or answers questions about their
shipping time for a drop shipping, Shopify marketplace, then you don't really need a sonnet or a
(07:53):
TFT 40 for those kinds of tests.
Yeah, exactly.
So I mean, like, for like, if I'm just like playing around, I want to use the latest and
greatest and know like what's possible.
But I think like, once you like kind of niche down onto your particular task, I mean,
like the smaller ones five.
I mean, I think that most of the cases just completely fine.
(08:14):
For the startup ecosystem at large, this is still fantastic news.
Yeah.
Yeah.
And you mentioned that that's kind of, I think Mark Zuckerberg is part of his strategy.
He wants people to start using llama just for all the things.
That's right.
He was talking about this again yesterday or a couple days ago.
He was at Nvidia's SIG graph conference.
(08:38):
And he was sitting with the Jensen Huang, the CEO of Nvidia and talking about some of their
high level strategies for tackling this problem.
And they were talking about how access to GPUs has always been a problem.
And Zuck was giving Jensen a little bit of, you know, a hard time for not delivering GPUs
(08:59):
on time and being a supply constrained.
Then talking about how open sourcing has worked really well for Nvidia in the past.
How a lot of the actually Nvidia or for one of them, how they open source some of the networking
switches and that became an industry standard and started being used by everyone else.
(09:22):
And then they were able to take all the improvements that the open source communities made to these
networking switches and then incorporated back into their architecture.
So they're planning to do something similar with llama as more and more people find bugs
and vulnerabilities in llama as they start using it themselves and make improvements.
They are able to, you know, absorb these improvements from the open source community
(09:45):
and then make the next version of llama bigger, better, faster, more secure.
That's true.
It really feels like some sort of virtuous flywheel for like Facebook where they're going to create
new bigger models.
Everybody's going to go.
So I think I remember seeing like the day after llama through poem was launched.
I saw like a couple hackathons just being posted like, oh, emergency hackathon.
(10:09):
We need to like build stuff on top of this llama 3.1.
It's exciting.
And now like if people start building stuff on llama 3.1, then like people are going to,
you know, kind of just be more used to like the Facebook stack.
And then they're going to become like the standard or because we build on top of it.
And then like then Facebook is going to get a bunch of stuff for free.
(10:30):
They're going to get the majority of the mindset or the developers using it.
And also like people are going to use them because they can run it like on prem.
So Facebook is going to get a bunch of free development work.
And then like I mean, Facebook isn't necessarily making money off these LLMs.
They're making money off of like showing ads.
So like if people build a bunch of cool stuff, then Facebook could probably show better ads
(10:54):
their users and then you know, make more money.
Facebook's docked to the moon.
Yeah.
And to make a small distinction, they're not open sourcing the product.
They're open sourcing the underlying technology behind the product.
So the product that they've integrated into WhatsApp and Facebook Messenger and a bunch
of other products is using llama 3 and lots of other rag type filters to pull context
(11:21):
from their social networks and your personal chat history, etc.
So they're adding a lot of things on top of llama 3 that they're going to keep close
source and maintain their competitive advantage.
Otherwise like they give everything out for free that'll kind of make them lose their
advantage for sure, for sure.
(11:42):
But you know, it really feels to me like it's becoming a race to the bottom and nobody is
going to be making money by selling these LLMs.
They're selling LLMs.
They're selling LLMs.
They've got a couple foundation models.
I don't know how competitive they are.
Oh, how good they are.
But everyone is selling foundation models.
(12:05):
But like they're actually selling the foundation model or they're giving away for free.
They might be giving it away for free.
Yeah.
But like the thing is, if you think about it, there's actually very few companies right now
that are making money directly from LLM, like LLM as the product.
I mean, the only ones that I can think of are on Thropic and Chatchapetit.
Google.
(12:26):
Oh, Google, right.
True.
Pi, character AI, but isn't Pi, Amazon has one for enterprise.
Yeah, but like I think meta AI, you can pay for it.
X AI, you can pay for it.
I actually meta, you can't pay for it.
Meta is free.
X AI, you can start paying for it.
Oh.
Oh.
(12:46):
Whoa.
Okay.
Anyways, let's get with that lesson.
All right.
Yeah.
Let's just ignore that.
So I guess you're right.
I guess there's like a few, but I don't know.
Like it seems to me that like all these companies are going to have a harder and harder time
justifying like $10 a month or $20 a month or whatever it is.
Yeah, you're right.
Like Google has like the Gemini Advance, Chatchapetit has their GPD 4 for sale.
(13:12):
Pi, is that free?
Oh, that was free.
It is free, but I do think they have a paid version too.
Okay.
So anyways, like the fact that all, a lot of these are going to be open source and people
can just run on their own.
I think the only reason why people may pay is just because they don't want to have to
(13:33):
host the model themselves and then want to use the latest and greatest.
But the fact that a lot of these smaller models are kind of good enough, as I mentioned
before, right?
Like a lot of people won't feel the need to pay like Facebook to run it because if they
have like a specific use case that they want to use the model for, they might just run
(13:57):
on their own hardware.
Well, that's why these companies need to do better and better with every release to make
it worth paying the subscription fee.
Otherwise, people are just going to turn to these cheaper models.
So OpenAI definitely keeps pushing the frontier.
They released Search GPT.
Should we talk about that?
(14:18):
We can.
I'm on the wait list now.
Yeah.
So, this is supposed to be the competitor to Google search.
And especially the Gemini summaries that you see at the top of Google search, if you have
that enabled or perplexity, which was dubbed the Google Killer, which uses these LLMs.
It's actually LLMs look agnostic.
(14:39):
They use whatever LLMs that you want.
You can be from Anthropic, ChatGPT, Google, and they use these LLMs to search the web, filter
the results, and summarize these search results.
And multiple steps if it needs to.
So it's cool that ChatGPT is doing this finally.
I feel like it was about time.
Because Search has been like the first place that people go to for a long time and to integrate
(15:02):
that with your LLMs.
That's fantastic.
It is a little interesting that Bing has been funding OpenAI for a long time and they
wanted to try to take over the search market with ChatGPT.
But now if ChatGPT has searched built in, are people really going to go to Bing?
(15:26):
I don't know.
Yeah.
And I also wonder if this is going to hurt Google.
Because if Google started to go down after that and find the set or something.
The thing is if people start just getting the answer from ChatGPT and they go to that
over Google, Google doesn't in my opinion have a super big moat.
(15:47):
I mean they have a big moat, but like it's not that big.
Well the people who work on Google search are some of the smartest people that I know and
we have the most technologically advanced stack to do any kind of search.
The latency that we have is insane.
I don't think ChatGPT can match like anywhere close to the latency that we get for both the
(16:11):
Google search and the Gemini results on top.
It's blazing fast.
That's true.
It is very fast.
But I don't know.
I think the thing is, at least for me I feel like a lot of times my Google search results aren't
exactly what I want.
Something I have to click through a lot of links.
(16:33):
I think there's a lot of SEO content.
I actually personally don't use Google search anymore.
You are one of the few outliers that you see to make that clear.
I mean that's true.
I am an outlier.
I don't use Google search at all.
I use a different search end called Kaggy.
So Kaggy like K, A, G, I, it's a paid search engine.
(16:55):
I paid ten bucks a month for it.
It's not sponsored.
Not sponsored.
I pay them money.
If they want to give me money to talk about them, that'd be fantastic.
Although they're a pretty small company.
But yeah, I think I personally prefer the results over a little bit.
Maybe I'm biased.
(17:19):
And actually Kaggy has a similar instant answer like Google search as well.
So it's got all the same things.
It should be cool.
They're probably using a smaller model.
To get that kind of latency on a bigger model would be hard.
But I feel like you're paying for it.
Maybe that subscription fee goes towards data centers.
(17:40):
Probably.
Yeah.
But I do feel like sometimes Google is a little bit slow for the instant answers.
So like it does buffer.
It doesn't render immediately.
Yeah.
So like for example, I was looking at some Android developer in my full time and my full
time job.
And I was looking through the Android developer documentation.
(18:02):
I wanted to look up some, you know, how to do some things.
And I saw that like Google started adding instant answers to their Android developer documentation.
And it was too slow to be useful.
So it also gave me the exact same result as what they just had lower in the page.
So it didn't really help.
(18:22):
But that's helpful.
I mean, I guess I don't have to scroll down the page or sift to all that content, which
is fine.
But I also use like command app and like find it immediately and then get like the actual
documentation and not like some summary of it.
So I don't know.
I mean, like, I think like the concept is cool, but I haven't found it like super useful yet
(18:46):
in my workflow.
Yeah, I don't know.
But I do think that the search, what do they call search GPT or I heard it's GPT.
Yeah.
It's exciting.
It's exciting and there's more competition out there.
So I welcome the competition and I know like I think Google's like still the top dog, but
(19:11):
there I think still is like room for somebody else to come in and definitely disrupt them.
Definitely.
This is a new world.
And I have been waiting for opening it to do this for a while.
I'm glad they finally did it.
And looking forward to it.
Yeah.
Yeah.
It's cool stuff.
So let's see what else is in the news.
(19:33):
You mentioned that, oh, actually here's something small, but pretty interesting.
Stability AI, they introduced a new thing called stable fast 3D.
Kind of cool.
So what it is, you can give it an image and it will generate a 3D model and it'll do it
(19:54):
really fast.
So I think you could probably use it.
I would imagine like the main thing would be like maybe video games.
So like you could take an image of like, I don't know, like a couch or like a cup or whatever
and then it'll go in and then like make a 3D model of that thing.
So where you used to have to you know, pay some sort of like graphic designer, I don't
(20:16):
know what you call like a 3D artist.
I don't know what like the name of that is digital artists, digital artists.
So anyways, you had to pay one of those people to you know, spend all day making your
different 3D models for the video game.
Now you could just like potentially, I mean honestly, you could probably do like a whole
end to end thing where maybe you have like an AI model.
So I could say like, hey, chat to the tea, create me a picture of a, I don't know, like
(20:40):
a car and then I could feed that car into this model and then now I have like a cool new
car for my video game.
So I think that we're going to start seeing like really good video games in the near future.
That is really cool.
I'm looking at their website for some more information and it's really cool how they take
(21:01):
the image and construct a 3D object with information about the what is behind the object.
Because oftentimes we get a lot of artifacts and like missing, um, depth and things like that.
But I think it has enough understanding of what you know, different kinds of common objects
are like a teddy bear or a sandwich or a couch where it can fill in the back in like a reasonable
(21:26):
way.
And they show us a show of video for like 4D outputs, which is like a 3D model with some
kind of animation mostly for characters.
So if it's like a dolphin, it'll be like flipping its tail or if it's like a camel, it'll
be walking, moving its limbs and head and it looks really cool.
Yeah, and also it seems like it's open weight.
(21:50):
That is surprising because stability has been going through a lot of turnover in their
leadership.
I think I'm odd, their founder, CEO left and they've been trying to figure out how to convert
from this whole open, free, everything approach to trying to make money and staying afloat is
very expensive to build and train all these models.
(22:13):
They've gone from image which they started with to language, which they added on later,
to video, now 3D.
That must be very, very expensive.
Yeah, I'm really curious like how a lot of these companies are going to stay in business.
Well, stability was just bankrupt, but this one guy who made a huge fortune and he's just
(22:35):
like, you know what, I want to give this to the world to kickstart the new generation of
hackers and independent developers to give them, you know, democratize this technology and
not keep it in the hands of open AI.
I mean, that's fantastic or mid-journey, which is the only competitive image generation
tool of the time.
(22:55):
Yeah, I mean, I think that's like great for the world, I fully support it.
The only thing is is like, I wonder if that's sustainable because I mean, clearly it's
not for them because the thing is is like, it seems like right now the only way you can
get these high quality models is by just throwing more compute at the problem, like more energy,
(23:16):
more GPUs, more data, just more.
And it seems like to scale.
And without any kind of like algorithmic advancements, it's just going to be only the biggest companies
in the world that are going to be able to train these models.
And I think it's even getting really expensive for the biggest companies.
I mean, think about like the biggest companies right now.
I mean, like, what's number one?
I think it's like, it's kind of sheer solid time like apples up there and video.
(23:41):
Video is close.
Yeah, actually not this week.
They've taken quite a hit.
Like Google's up there, Amazon, you know, whatever, like all these big companies.
If we're talking about individuals, I think Elon's up there too.
If we're talking about like entities who have a lot of money to throw the problem.
Yeah.
So his new X AI venture, which is trying to, you know, come up with a new LLM and multi-modal,
(24:06):
probably model.
He is trying to build a data center in Memphis and get access to close to 150 megawatts of energy.
And we were talking about this earlier for context.
I think your average home, you know, maybe like a three bedroom home would be in the low,
(24:28):
maybe 10 digit, sorry, it's like 10 kilowatts.
10 kilowatts, yeah.
Not 10s of kilowatts.
10 to be like a big home, I think.
10 kilowatts.
I think it's like two kilowatts.
What I thought, yeah, two, like a small studio one bedroom or something.
Yeah.
And then like 10 to be like a higher-sager, just yeah.
And so he's raising like a hundred thousand times or 10,000 times, he's looking for 10,000
(24:53):
times more energy than what a single family home would be using.
Seems big.
Do you know any reason why he chose Memphis?
No idea.
Yeah.
Interesting.
I mean, like there's nothing wrong with Memphis.
Big fan of Memphis, great place.
I'm just curious because like, you know, all of his companies are in like California or Texas.
(25:15):
So like, Memphis seems interesting.
He had a big gigafactory in Nevada too.
I think the way he approaches these problems is goes to the person who can offer the most
value or instead of person, it would be government entity state.
Maybe there's access to cheap energy.
(25:35):
Maybe there's big subsidies.
So for this particular decision, it seems like Tennessee offered ample access to power
and the ability to build quickly.
Maybe removing some of the red tape.
That seems good.
Yeah.
I mean, I guess it doesn't really matter where you build it because like all that really matters
(25:58):
is like, I think the main thing would be like you want an area for going to build a data
center.
You probably want it to have like low natural, low rates of natural disaster, political
stability, political stability, public benefit, water to cool all these data centers.
Yeah.
Cheap energy.
(26:19):
I don't know, just some, but it doesn't need to be like in Manhattan or something like that.
Yeah, because you're training these models.
You're not making them available.
You can make them available in other data centers close to metros.
Right.
Exactly.
I mean, like Memphis, I mean, that's still like a decently sized metro that is close to
(26:41):
a lot of people.
So that's good.
Maybe.
I mean, yeah, there's a lot of people living in Memphis.
It's a good place.
I mean, you want to cover the West Coast and East Coast though if you're talking about the
US.
Well, I mean, maybe Chicago.
You're just training it.
You don't need to make it available.
Yeah.
So I mean, like it could be anywhere.
You can do this in Alaska.
(27:01):
It doesn't matter.
Yeah, that's fair.
Like, it's fine.
I could go and make it.
It doesn't matter.
You could.
I mean, just all you need is like a nice little data cable that goes from Antarctica to,
like, I don't know, the closest city, like somewhere like in Alaska.
You don't need cables.
He's got Starlink.
Beam it all into the atmosphere.
Oh, yay.
Oh, actually, I was thinking, I don't know what the throughput of that would be though.
(27:25):
Good enough.
I don't know.
I mean, like, the model aren't that big.
A lot of data though.
That's true.
That's true.
How would you get the data to Antarctica?
Just a container ship full of hard drives.
You could actually, you should never underestimate the power of, like, semi-trucks, like, filled
(27:47):
with hard drives, like, barreling down the highway.
I'm sure they're using hard drives to back up some of these models.
I mean, why wouldn't you?
I mean, like, we're also going to put it tapes.
But tapes are kind of slow.
They are racing.
Yeah.
Archival storage.
Yeah.
I mean, like, you could, I guess.
They do.
(28:07):
Yeah.
If I was going to do it, I would just probably use, like, hard drives.
I mean, because, like, the models aren't like, I don't actually know how much data they
use.
I imagine it's a lot.
But I think the thing is, is, like, I think it's not the data that's necessarily the bottleneck.
It's more like the compute.
Because, like, you need to, like, process the data, like, many times, and, like, lots of different
(28:33):
ways.
So, it's like, I think that, like, probably, like, honestly, just, like, a few, like, hard drives,
like, oh, maybe, like, a petabyte of data.
I mean, that's not, like, how many hard drives do you need for a petabyte data?
Because if you have, like, petabytes, like, a thousand terabytes, right?
So, I mean, you get, like, a one terabyte, I think they're, like, ten terabytes drives now.
(28:54):
So, I mean, what do you need?
Like, a hundred of those?
I mean, it's not that much space.
I mean, I could probably fit that in, like, let's read them.
The back of, like, you know, like a standard car.
It's not like, you guys are crazy, right?
Right.
Data isn't the, like, bottom, like, it is energy.
I don't think Elon's going to get all this energy immediately.
There have been a couple articles about how a lot of companies are freezing energy shortages
(29:18):
to train these models.
It's getting more expensive.
And it's like, especially Facebook, the way they have improved the performance for some
of their models is just by training it longer.
So, I think maybe five years ago or something, they thought that there was a limit to how
good these models can get.
(29:38):
And it plateaus, which is kind of true.
It does appear to plateau, but it doesn't actually plateau.
It just improves much slower, the longer you train it.
So that's what Facebook has been doing.
They've just been continually training it for longer and longer, even though it doesn't
improve as much, but it does improve, like, very miscule amounts.
So basically, you're saying, just throwing money at the problem and then just making it
(30:03):
about as good as you can get it.
Keep making it better.
Keep making it better.
Keep throwing more money at it.
Yeah.
I feel like, is that sustainable?
Are we going to start to have nation states be the only types of people who can train these
models?
I mean, at a certain point, it's going to be none of the big companies are going to be
able to afford it.
And the only thing you're going to have is the US government, the Chinese government,
(30:27):
it's like people who are lobby should be allocated money to healthcare or training the next model.
It might be.
I mean, you could like legitimately see that future where it's only like the few largest
governments on earth.
Because I mean, the thing is, there's like a lot of these big mega companies.
I think they're being stretched.
(30:48):
And there's nobody who has more money than that.
I mean, like what country has more money than like Facebook?
It's like this very few.
Obviously the one that we're in right now.
Well, I mean, like the United States, sure, like China, Japan, Germany, Germany, right?
(31:10):
But like, I mean, once you get past the top 10, I mean, there's not many countries that
have like more money than those companies.
Sure.
So these companies are massive, yeah.
Right?
I mean, these companies are like so rich and valuable that like no one will be able to do
it.
And then it's just going to become like a government thing like a like who knows is like maybe
(31:34):
we'll start seeing these models are going to be like water or it's like it's just going
to not be like PG&E where you get your water or like your electricity.
It'll just be like you're going to pay like your monthly fee and get your model.
It's like you know, regulated by the government.
It'll just be like a pseudo government entity kind of like honestly, be like the internet
(31:56):
almost like I feel like the internet is almost like an equivalent to telecom or railroads
or some fundamental infrastructure.
Right.
Right.
It'll be fun about the infrastructure.
It's an outer electricity.
Yeah.
I mean, I don't see why it wouldn't be.
It feels like it's going that way.
But I mean, at least in the US, a lot of these things are privatized.
(32:18):
PG&E is a private.
Yeah, it's like private technically.
Yeah, private it.
And I mean like same with like the telecom companies like I mean like what are the big
ones in the US like 18T for rise T mobile.
I think it's consolidated into those two.
Yeah.
Three right.
(32:38):
T mobile as well.
But separate.
Yeah.
But I mean, the thing is is like they're not like the government.
But like they're highly regulated.
Highly regulated.
Yeah.
Same with PG&E like I know that like technically private company, but it's like highly regulated.
Like I mean, the government has essentially like given them monopolies and like I would
(33:02):
not be surprised if even though like open AI is like a private company that's like pseudo
owned by Microsoft right now.
Like I would not be surprised if they just start being like highly regulated by the government
and it becomes like almost like another R of the government.
(33:25):
Like maybe we'll start seem like the biggest models being controlled by like I don't know
like the US military or something.
I could see it being similar to Uber.
How they faced a lot of regulation.
They're still a very private company, but they do have a lot of government oversight.
And each country stayed that they operate in has its own set of rules and regulations that
(33:45):
it has to comply with.
Yeah.
I mean, but the thing is I think the difference with Uber is like there isn't any like fundamental
reason why people can't compete with them.
Like it's not like they process so much data.
I mean, they're like a big network, right?
They're like so regulated, but I think that like they're regulated in order of magnitude
less than like the utility company.
(34:07):
Sure.
So that's why I would sort of think that like like it's a big if, but if these model like
if we don't see like major algorithmic changes and if these models, you know, let's say like
in five years from now, it takes like a trillion dollars to train them, but in these new models.
(34:29):
If like if it just happens that the only way to make these models better is by just throwing
more data at the problem, more energy, more money.
But only entities are going to be able to do it are going to be governments.
And then like at that point, maybe it was give it away to like the masses and like you just
pay for it like a utility bill.
I don't know.
I can see it happen.
Yeah, interesting problem because like now that all these models are being given away
(34:53):
for free, how do they make money?
It's like apart from their main business, you know, ads for Google and ads for Facebook,
okay, there's to figure out how to make money from these things. And especially if video generation
takes off, we still don't have like widespread video generation tools available.
It's been advertised.
I think Google also announced it's new Vio model or DeepMind did and I'm on the wait list.
(35:20):
I just signed up.
But once that takes off video generation training these models is going to be very computationally
intensive.
Yeah, and also just like as people use these models more for just like regular things, right?
Like once we see like more agent work flows, once we see more people just like integrate
(35:41):
them into apps, just they get more popular.
That's just going to like really increase the need of computing.
I think maybe at some point we'll just start needing to use like a license for you to
like the energy that we need.
Yeah.
So we're still along with a long ways away from there.
Not that far away from the distance.
You never know.
(36:02):
Maybe the exponential growth is closer than we think.
It is closer than you think.
I mean, yeah, it's like as they say, if you have like a like bacteria in a lake that's
doubling every, you know, every day, right?
You can't handle that many double links, right?
(36:22):
Double links is a dangerous dangerous phenomenon.
And I mean, like the computing power is doubling every two years.
Yeah.
So I mean, we're going to need it soon enough.
Dyson sphere.
All right.
Let's start planning for the Dyson sphere then.
All right.
Sounds good.
I know we're just about out of time.
But anyways, all right guys, if you have any like ideas on how we can avoid the Dyson
(36:45):
sphere and just like make the models better, let us know.
We'll work on it.
So we'll see you in the next one until next time.
Popular Podcasts
United States of Kennedy
United States of Kennedy is a podcast about our cultural fascination with the Kennedy dynasty. Every week, hosts Lyra Smith and George Civeris go into one aspect of the Kennedy story.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Bookmarked by Reese's Book Club
Welcome to Bookmarked by Reese’s Book Club — the podcast where great stories, bold women, and irresistible conversations collide! Hosted by award-winning journalist Danielle Robay, each week new episodes balance thoughtful literary insight with the fervor of buzzy book trends, pop culture and more. Bookmarked brings together celebrities, tastemakers, influencers and authors from Reese's Book Club and beyond to share stories that transcend the page. Pull up a chair. You’re not just listening — you’re part of the conversation.