July 7, 2024 • 109 mins
Our 173rd episode with a summary and discussion of last week's big AI news!
With hosts Andrey Kurenkov (https://twitter.com/andrey_kurenkov) and Jeremie Harris (https://twitter.com/jeremiecharris)
See full episode notes here.
Read out our text newsletter and comment on the podcast at https://lastweekin.ai/
If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form.
Email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai
In this episode of Last Week in AI, we explore the latest advancements and debates in the AI field, including Google's release of Gemini 1.5, Meta's upcoming LLaMA 3, and Runway's Gen 3 Alpha video model. We discuss emerging AI features, legal disputes over data usage, and China's competition in AI. The conversation spans innovative research developments, cost considerations of AI architectures, and policy changes like the U.S. Supreme Court striking down Chevron deference. We also cover U.S. export controls on AI chips to China, workforce development in the semiconductor industry, and Bridgewater's new AI-driven financial fund, evaluating the broader financial and regulatory impacts of AI technologies.Timestamps + links:
- (00:00:00) Intro / Banter
- Tools & Apps
- (00:03:24) Google opens up Gemini 1.5 Flash, Pro with 2M tokens to the public
- (00:08:47) Meta is about to launch its biggest Llama model yet — here’s why it’s a big deal
- (00:12:38) Runway’s Gen-3 Alpha AI video model now available – but there’s a catch
- (00:16:28) This is Google AI, and it's coming to the Pixel 9
- (00:17:30) AI Firm ElevenLabs Sets Audio Reader Pact With Judy Garland, James Dean, Burt Reynolds and Laurence Olivier Estates
- (00:20:06) Perplexity’s ‘Pro Search’ AI upgrade makes it better at math and research
- (00:23:12) Gemini’s data-analyzing abilities aren’t as good as Google claims
- Applications & Business
- (00:26:38) Quora’s Chatbot Platform Poe Allows Users to Download Paywalled Articles on Demand
- (00:32:04) Huawei and Wuhan Xinxin to develop high-bandwidth memory chips amid US restrictions
- (00:34:57)
(00:05):
Gen 3 Alpha 2, so fun.
Get ready for a wild ride.
We'll
make it fun, you see.
(00:25):
From AI agents to Chevron deference with Glee.
Hello and welcome to the latest episode ofLast Week in AI, our podcast where you can
hear us chat about what's going on with AI.
And as usual, in this episode, we will summarize anddiscuss some of last week's most interesting AI news.
(00:48):
And as always, you can also check out ourLast Week in AI newsletter at lastweekin.
ai for articles we did not cover in this episode.
I am one of your hosts, Andrey Kurenkov.
For some context, I did a PhD in AI atStanford that I finished last year, and
I now work at a generative AI startup.
And I'm your other host, Jeremy Harris.
(01:09):
I'm the co founder and CEO of a company called Gladstone AI.
We do
And, you know, we've had a bunch of really useful feedbacklately on the podcast on a number of different things.
One of them was that apparently thesound coming from my end was shit.
And so we're going to try to fix thattoday, um, in a couple of different ways.
(01:30):
I'm register.
I'm recording locally, but I alsomove my microphone closer to my face.
And so, uh, for those of you who are watching onYouTube, uh, you probably like that because it
takes up a lot of the space that otherwise my facewould in the shot, which is probably a good thing.
Um, anyway, so, uh, gonna try to , get youguys some better sound quality this time around
and hopefully, uh, a bit of a smoother ride.
(01:50):
Um, so yeah, I
actually like it in frame because it makes it feel morelike a professional podcast as opposed to what it is,
which is like two guys full time jobs just doing this onthe side with one of them, like fully a non audio engineer
taking care of editing, but you know, we try our bestto actually, uh, shout out to the person who Point to
(02:12):
yourself, Jenga rocks on Apple podcasts, got us a threestar review, which is not great, but very useful feedback.
Uh, and we will try our best to takeit into account and, uh, do better.
We'll see if, uh, that actually works.
Plus we look more
like Larry King this way.
Um, which, you know, Plays into our ongoing strategyto gradually turn into Larry King, because that is.
(02:38):
thing that people would want to do for some reason.
Okay.
News to me that that's the goal, but,uh, I guess we'll see if that works out.
I thought we were completely aligned on that, but okay.
And real quick, just another shout out to another review.
Uh, green poo, VFO podcasts, uh, has areview titled keep the doom, keep the doom.
(03:03):
Okay.
Yes.
Uh, and, uh, yeah, it says keep with doomsday stuff, please.
I'm not sure how this became a whole saga of I think
there's a war playing out right now in our colony region.
And yeah.
Yeah.
Yeah.
So
we'll try to appease everyone.
Maybe a little bit of doom every now and then.
Just a
little bit of doom.
Yeah.
Yep.
(03:23):
And that's the news starting with tools and apps.
The first story is Google opens up Gemini 1.
5 flash and pro with pro having2 million tokens to the public.
So this is in Google cloud and these two modelsare now publicly accessible with Gemini 1.
5 flash being a small multimodal model that, uh,is more for high frequency tasks and Gemini Bye.
(03:52):
Pro is the most powerful version of theirGemini LLM, and notably has a 2 million
context window, 2 million token context window.
That's the amount you can have in the input.
And for some context, that is a lot.
That's like 60, 000 lines of code over 1.
(04:14):
5 million words.
So this is one of the bigsurprises to me over the past year.
You know, if you remember in the early days of GP3,the inputs we could get are like 000, relatively small.
And I felt that this would be one of thefundamental limitations of a transformer model.
(04:34):
And it turned out to be not the case at all.
So there you go.
It's a certainly an exciting day for developers.
And, uh, these are some pretty capable models.
So pretty nice to have.
And they're really starting to compete onthose those sort of secondary metrics, right?
We all often talk about, like you said, context, windowlength, you talk about, you know, sometimes perplexity
(04:56):
or whatever, but we're starting to see stuff at theengineering level really become the main point of
comparison is inference gets more and more important.
So here in particular, you know, they'll tell you, yeah.
Gemini 1.
5 flash.
Um, is they say 40 percent faster than GPT 3.
5 turbo.
That's at an input input of about 10,000 characters, which seems pretty fair.
Um, so, you know, that's basically saying,Hey, we have 40 percent lower latency.
(05:20):
They also apparently have a, an input price per token price.
That's four times lower than a GPT 3.
5 with, um, with this thing called context cachingenabled for These larger inputs, which is its own kind
of interesting sub announcement in this whole thing.
They're launching this, um, what they'recalled this context caching feature.
(05:41):
Uh, as a public preview right now, thisis both for flash and pro at Gemini 1.
5.
This is basically a thing that allowsmodels to store and reuse information.
They've already.
Um, kind of processed without having to recompute everythingfrom scratch every single time they get a request, right?
So think about like your sort of short term memory.
Um, I assume that they're using justkind of embeddings to achieve this.
So it's like kind of a middle ground with rag.
(06:03):
Um, anyway, so, so really interesting.
Another cost saving strategy, keep the thingsin context to not have to reprocess them.
That all seems to be part of the, sort of the, um,turning this into more of an engineering, not more
of an engineering discipline, I guess, but more of auser facing productized thing rather than a research
artifact that we're seeing across the industry right now.
Yeah.
(06:23):
That's a very good call out.
I think this caching feature is.
More of a big deal than it might seem.
To me, it's been a bit surprisingthat it hasn't already been the case.
Uh, the idea here is if you're using kind of fine tuningthese LLMs via just a system prompt, you condition
them to say, this is your task, blah, blah, blah, blah.
(06:44):
If you just reuse that without any modification, itmakes a lot of sense that you're able to cash with.
And, uh, I actually looked around.
There aren't, uh.
except for this, uh, existing cloudmodels that allow you to do this.
So probably pretty useful.
One other note that also is in the story,apparently there was a press briefing in which the
(07:06):
Google cloud chief executive said that a lot oforganizations, big ones like Accenture, Airbus, Box,
Broadcom, et cetera, are building on the platform.
And this is one thing also notto discount that, you know, they.
Google may have seemed to be behind OpenAI indeveloping Gemini, but they are a big company
(07:31):
that is certainly ahead in terms of being able toprovide enterprise offerings on its cloud platform.
So it makes a lot of sense that these kind ofbig companies would work with Google, potentially
Microsoft and Azure as well, but certainly Googledoes have an edge here that may be, uh, not apparent.
Yeah, you know, they often talk about,uh, the importance of distribution, right?
(07:54):
As, as the central thing, the, the, theexample I always remember from back when
I, you know, we're doing the YC stuff.
I think Microsoft Teams, uh, versus Slackis just such a great example, right?
Slack launches, they get this likeslow, steady increase in user traffic.
I mean, it's very fast, right?
This is a successful Silicon Valley company.
But, um, you know, Microsoft teams launches andthey're like, their stick looks like a hockey, sorry,
(08:18):
their graph looks like a hockey stick in comparisonbecause they start with all this insane distribution.
And, you know, you see open AI trying to kindof make up for that deficit by partnering
with Apple, by partnering with Microsoft.
Um, but ultimately Google has that baked in, right?
So this is a structural advantagethat they enjoy over open AI.
Much more so.
So kind of interesting.
Yeah.
That, that this is something they don'thave to worry quite so much about.
(08:40):
It is an access they can use to, to differentiateon and just kind of like force users to
start using their products essentially.
And next of a big company, Meta is about to launch itsbiggest llama model yet, which is a llama free phone.
400 billion.
So when Llama 3 came out with the 8 billion and70 billion parameter sizes, we also got a bit of a
(09:04):
preview, uh, with the awareness that this 400 billionparameter exists, that it is GPT 4 kind of level.
model that they were still training,which was kind of a funny note that I had.
And they showed that on the majorbenchmarks, uh, it was matching GPT 4.
(09:25):
So apparently it's nearing releasenow, a couple months later.
We don't know if it will actually be released and opensourced the same way as the smaller, uh, sizes have been,
but it does seem like now it's being rolled out in someof the meta products such as on WhatsApp for beta users.
(09:46):
It's now, there's an option touse the Llama3 405 billion model.
Yeah.
And these use cases, even these really beta onesdo seem to have significant limits on usage volume.
Um, so, you know, just to kind of get people stoked aboutthe upcoming release, this article is speculating July
or August of this year, you know, we'll have to wait andsee one of the, the, um, things I suspect that they're
(10:08):
going to be struggling with here is making sure thatthe model can't be jailbroken as trivially as the other
Lama three models can be, uh, you know, this has been apersistent issue for meta as they've in some cases been
embarrassed by, uh, Like congressional testimony, youknow, Zuck got dragged in front of some committee or other.
And I think, um, I'm trying to remember if TristanHarris or Rio, who, who it was exactly, who sort of held
(10:30):
up this jailbroken version of llama too, at the time.
And, you know, Zuck was saying, ah, well, youknow, you can't, can't do these bad things.
And I said, well, look, we took your, yourmodel and basically like trained it just a
little bit and it forgot all of its safeguards.
And now we can, you know, get it to try at least todo some bad things, even if it couldn't successfully.
Execute them due to a lack of capabilities.
So I think this is going to be a big issuebecause when you start to think about models
(10:52):
that are like llama three, 400 billion, right?
This is a model, as you rightly said, that's beencompared internally by meta to GPT four level performance.
Well, you know, what are the thingsthat we know GPT four can do it?
There's things like.
autonomously discovering and exploiting, you know,one day vulnerabilities, zero day vulnerabilities,
um, significantly leading a significant measurablelifts in cyber offense capabilities, even to
(11:14):
some degree on a bioweapon design capabilities.
And so we're already there.
You start to open source thosesystems that can create some problems.
So internally, I do wonder what that discussion sounds like.
I wonder what kinds of safeguards They're planningto implement because it's not even just like, Hey,
let's implement safeguards that are robust to thejailbreaking techniques that we know of today.
They have to be robust to jailbreakingtechniques that have not yet been invented.
(11:37):
And it's really not clear, youknow, how you square that circle.
So I love to be a fly on the wall right now in the kind of,in the red teaming rooms at meta and the kind of decision
rooms where they're thinking about, you know, the prosand cons of releasing that 400 billion parameter model.
But if it comes out, it'll be a very, very big deal.
Yeah, for sure.
And I think there's really no way given whatwe know about jailbreaks, if you're able to
(11:57):
train the model, you can't safeguard against it.
It's just impossible.
So this is one of those questions that has beendiscussed a lot in open source is what's the benefit
versus the risk of releasing models and enablinguh, actors to misuse them to nefarious ends.
And so we'll have to see whatconsideration meta makes on this one.
(12:21):
Uh, and you know, if it's released, maybe that'sa doomsday scenario, kind of the conversation from
there, but you know what?
Maybe, maybe the 4 trillion, uh, predator.
Yeah.
The next one.
Yeah, it's almost a 10x away from here, right?
That's what they, uh, they always say.
Onto the lightning round.
First story is Runway's Gen 3 AlphaAI video model is now available.
(12:46):
So this was announced and previewed very recently.
I think we covered the kind of announcementand some of our preview videos last week.
They said that it would be available to users of theirproducts very soon, and that has turned out to be the case.
It is now available to paying users.
So you have to pay 12 a month to be able to access a model.
(13:09):
You will now be able to generate videos up to10 seconds and which generation speed will vary.
Uh, and it's not quite solar level yet,but it, it is a big upgrade over gen two.
And there aren't many players thathave launched these sort of tools.
We've seen Luma do that, but thatwas sort of a public preview.
(13:30):
We don't have many customers yet.
To my awareness.
So, uh, runway has been a game for a long time.
This is a pretty impressive modelthey've caught up in some sense.
So I wouldn't discount them even as Soraand things like that are coming out.
And like you said, I mean, there areother competitors entering the space.
You mentioned Luma stability, obviously Pika openingeye itself with Sora though that hasn't been released.
(13:50):
So, you know, the space is heating up a little bit.
They, they had been working pretty quickly back inthe gen one gen two days runway had that is, um,
you know, pumping out those models pretty quickly.
Absolutely.
A few months between them, um, the, thebig kind of gap that we've seen is really
when all these extra players have come in.
So I think that's, you know, now people are startingto ask those fundamental business questions,
(14:11):
especially looking at the situation with stability,where this, you know, open source, everything.
A story just did not seem to pan out, you know, Andre,I remember we were talking about that back in the day
being like, we have, what is the business model here?
Like at some point you got to make somedollars and I get that you can do it in various
interesting ways by hosting and all that stuff.
But, but you know, at a certain point you need acompetitive moat and, um, it looks like, you know,
(14:32):
again, offering some users access to paid versions ofthese products does seem like it's an obvious default.
Um, so we'll see if that becomes a norm more soon the kind of generative video and image stuff.
But, uh, one thing that they do say is that.
That the, the model is, uh, currently runs withtext to video, obviously, but in the coming
days they'll be, uh, releasing other moats.
(14:52):
So that'll include apparentlyimage to video and video to video.
That's gonna be really interesting, you know, like whatnew editing possibilities, uh, are accessible as well.
Right now they are capped at 10seconds of video generation, right?
So, you know, SOA famously is, is aminute long, so this is 10 seconds.
Um, but then again, you know, so hard to compareit to Sora cause like we don't have access to it.
(15:13):
It hasn't been released for general use.
So we'll just have to see, but, um, definitely, you know,big step on the path to productization of text to video.
This is a big day.
One thing to note about Runway as someone who hasplayed around with it and has kept track of a company,
uh, it's worth knowing that they offer a lot of stuff.
stuff aside from the video generation, they offer backgroundremoval, uh, kind of optimization of video, up rezzing,
(15:38):
all these things that as a professional in the industry.
Most likely you would get a lot of benefit from beyond videogeneration, which is pretty unproven and is like a cool
thing to fundraise and get money from investors for, butnot necessarily something that people need at the moment.
So, uh, I don't know.
I don't know why I want to keepsaying this, but I'm pretty sure.
(16:00):
Kind of a fan of Runway and I, uh, like tosee them having these tools that seem like
they would have a pretty significant impact.
And then, uh, yeah, as you'll cover, theyare hoping to fundraise quite a bit soon.
And you, uh, use it at yourcurrent job, like the video game.
I don't use it.
I've played a round of it.
(16:21):
I'm always curious with, with the, the workthat you're doing there, you know, to know,
well, like what tools are actually useful whenyou're doing generative AI for video games.
So yeah.
That's it.
That's cool.
And next we are getting to Google as yet anotherbig company and the news there is that there is
some AI that's coming to their Pixel 9 smartphone.
So this is basically a set of variousfeatures under a branding of Google AI.
(16:45):
ai things like add Me that ensures, uh, everyoneis in a group photo studio, which is, uh, AI image
generation pixel screenshot that ev you know, evidentlywill be similar to recall, where you can take a
screenshot of a phone and talk to the AI about that.
(17:05):
So these various built in things comingto Google phones in a way I think.
Reminiscent of some of the announcements ofApple building stuff into iOS and we'll see how
much people like it or if they just ignore it.
Yeah,
and you know, this is google continuing to tryto figure out how do we integrate everything
(17:26):
under one under one hood here, right?
So, uh, yeah, we'll we'll see if it's next
Next, AI firm 11Labs sets Audio Reader packed with JudyGarland, James Dean, Bert Reynolds, and Laurence Olivier.
11Labs is the audio generation firm thathas allowed people to Turn text to voice and
(17:50):
recently they announced this reader app, whichessentially provides narration for any text.
Now it seems that they have had these agreements to havethe voices of these famous actresses and actors to do that.
Uh, reading for you, they say this would be partof the iconic voices feature of the, so I don't
(18:17):
know, interesting, you know, uh, apparently thishasn't been met with quite as much controversy.
It was a pretty legit deal, a licensing deal.
The daughter of Judy Garland has endorsed the deal.
Uh, so.
I think you as a user, right?
You want sort of a professional, anarrator rather than a robody voice.
(18:40):
So certainly I could see why people might want to use this.
Yeah, no, for sure.
It's interesting.
This is almost like the soft underbelly of how to get, youknow, automated, um, text to audio done without Without
pissing people off, uh, while still really having thesame effect that a lot of people are complaining about,
you know, I think we talked about this in the contextof, um, a couple, I guess, like almost two years ago now.
(19:02):
So I, I wrote this book and I didthe, uh, the audio book for it.
And it, like, it was, Time consuming.
You're sitting in the studio.
It's like 16 hours of painful recording.
And if you don't do it as the author, someone else does.
There's a whole industry of people who will read thekind of text of a book and, and, you know, do the audio.
And people were saying, well, there was a scandal.
(19:22):
I think at the time we were talking about a scandal where,um, you know, companies were selling access to those audio
books, those, the sound that was being read by, uh, thoseon contract readers and saying, Hey, you can't use their.
Their own, you know, their own work, uh,to automate away their, their whole jobs.
Well, this is a way around that, right?
You go to people who, uh, aren't going to be producingany more work like that because they're already dead.
(19:46):
You go to their estates.
That's one good way to do it.
Um, but it achieves roughly the same end.
So, you know, uh, maybe not as, as controversial,but again, achieving the same ends and, um, we'll see
if this becomes a, a norm, you know, I, I think justgiven the cost of doing a lot of these recordings,
the whole studio time, multiple employees, soundengineers, like this, this could really be a thing.
(20:06):
Next perplexities pro search AI upgrade makesit better at math and research perplexities.
Of course, the AI powered search engine that,uh, basically you can use, uh, input a query.
It does search.
Online and looks up various websites andit provides an answer informed by that.
(20:27):
So they have now upgraded the pro search feature,which Would be the thing that you have to pay for and
there's a couple things in there They uh now have anupdated pages feature, which we covered a little while
ago And here they say that uh, it's going to be betterable to understand complex queries, plan responses, and
(20:50):
synthesize in depth answers outside of that pages feature.
Uh, and, uh, interesting.
Yeah.
Perplexity is one of the, you know,pretty buzzed about, uh, companies.
I think Jetson Huang said he uses it daily.
I've played around with it and Iwould say it seems quite useful.
So it makes sense for them to keep kind of.
(21:13):
Pressing the advantage we have in this space.
It's also the kind of
thing that is so much easier for perplexity topull off because they don't have all the baggage
of the sort of legacy, uh, use cases that acompany like Google, which nominally they frame
themselves up as competing directly with, uh, has.
And so they're able to just sort of be.
Excuse me, be more, um, through open horizonabout what, what their product possibilities are.
(21:37):
Um, just looking at the demo that they've got,you know, it, yeah, it looks pretty compelling.
They type in a search query, likeI want to see the Northern lights.
When is the best time to go?
And what are the top viewinglocations in Iceland or Finland?
And yeah, click on the pro search thing.
And it's Automatically goes through like, like any,you know, good web agent would to, uh, to dig up
the answer to your question in a way that's a littlebit richer and more detailed, uh, than necessarily
(22:01):
you would get through a standard search process.
So, yeah, we'll, we'll see if thisis actually the use case people want.
One of the things that does, you know, If you justjump to mind when you start to introduce this kind of
use case is again, inference becomes so, or sorry, uh,inference latency becomes so much more important, right?
You're just watching as this thingkind of thinks about your question.
And as I imagine myself as a user, right?
(22:22):
My, my u like my user experience is that of like.
Just hanging on while this thing chugs away, uh, thatmay or may not be what I want, but, uh, as you start
to see the, you know, the efficiency of inference, thecost of inference and the time of it go down, uh, you
know, these things might become more and more attractive.
So perplexity is definitely positioning itself throughthis to, uh, do some, some really interesting things.
(22:44):
And just FYI, if you recall just now, if youremember the silly things that Google's AI responses
did recommending people eat rocks, well, peopledid also check out what perplexity does there.
And surprise, surprise, perplexitydid not recommend bacteria trucks.
It actually recognized that it was silly andpointed out that there are some sarcastic responses.
(23:07):
So, yeah, it, it is a prettypolished product in this category.
And last story for the section, Gemini's dataanalyzing abilities aren't as good as Google claims.
These are some studies that showed that for largedata sets, the correctness rate is fairly low.
Apparently they give the right answeronly 40 to 50 percent of the time.
(23:32):
The tests here are basically evaluatingtrue false statements about fiction books.
And for these large Portions of a book which hasimplicit information that is not stated in the text.
Apparently, these models aren't that great.
And this is one of the kind of questions that arestill out there for these huge context windows.
(23:56):
How good are they in practice with giant inputs?
I don't think we know the answer yet, really.
And the studies do shed some light on that.
Yeah, there's some important framing here too,which is that it's not here that, you know, Google
is behind the curve or they're, they're, you know,they're doing things, um, badly on a technical level.
(24:16):
The complaint here really is about what Google'smarketing department has been doing, um,
basically over promising, under delivering.
I mean, Google's.
Obviously delivering excellent products.
Gemini 1.
5 pro 1.
5 flash.
These are really impressive models.
Uh, the challenge is that, you know, when you goout there and say, Hey, we, we can actually solve
these very complex data analytics problems, you know,maybe not, not quite the case or cherry pick things.
(24:37):
It's something we've seen with Google before, right?
We've covered stories where Google has come out withan impressive demo that doesn't quite seem to be.
It kind of live up to the hype.
Um, that's maybe a bit of a, an internal issue for them.
The other right hand, not knowing what theleft hand is doing perhaps, or the marketing
department just kind of getting ahead of itself.
Um, one of the things that does come across inthese, in these studies, it's kind of interesting,
(24:57):
you know, the, uh, the old needle in a haystack.
eval, which is the way very often these modelsability to like, um, recall information out of a
really, really large body of text is tested, um, isto insert a very specific piece of information, like
one sentence that is like out of place somewhere ina really long piece of text and testing to see if the
(25:19):
model can actually recover that sentence correctly.
And what they're pointing out here is.
That they notice, and this is a quote from the article,we've noticed that the models have more difficulty
verifying claims that require considering larger portionsof the book, or even the entire book, compared to claims
that can be solved by retrieving sentence level evidence.
So when the information you're trying to recover ismore diffuse, when you have to put a lot of things
(25:44):
together, all of a sudden your accuracy goes down a lot.
It's not a pure recall test.
It's a recall plus synthesis test.
And I think that's a really thing that highlights,you know, a kind of eval that we haven't seen
done, certainly not nearly enough of whenwe look at these big models in long context.
Yeah.
And I think it makes sense.
We've seen, uh, kind of in very earlier days of largecontext windows on fropics show that you could do
(26:08):
really well on these sorts of tests by just appending,you know, present the most relevant sentence, uh,
in the prompt for these needle and the haystack.
Benchmarks and it has been sort of criticizedthat yes, it's a needle in the haystack, but it's
a very obvious needle that really stands out.
(26:28):
So it, uh, yeah, these benchmarks, I think, haven't reallyshown how useful these context windows will be in practice.
And this sort of research is very informative.
Onto applications and business.
The first story is that Quora's chatbot platform Poeallows users to download paywalled articles on demand.
(26:49):
So you can download HTML files for articles publishedby paywalled journalistic outlets that include includes
things like the New York times, Bloomberg businessweek, Vietnantic, and all of these other big services.
We saw.
Also, the claim being made that Perplexity doesthis for Forbes, where they look at the article
(27:14):
and can produce kind of verbatim responses withoutreally citing, citing the source, but not that much.
And in this case, it also appears to be a case aboutthe bot does not appear to adhere to the robots
exclusion protocol, which is a web standard forprevent spots from accessing parts of websites.
(27:36):
So really interesting developments in this front wherethe internet and any publisher of information such as, uh,
journalists and things like Reddit, Twitter are now reallyhaving to consider, uh, blocking downloads of data, which
we've had paywalling for users, but it has been fairlyeasy to get around those paywalls if you tried at all.
(27:59):
And I think that's going to change very much.
Yeah, absolutely.
And it's really interesting to see thelegal arguments that are being made to say
like, Hey, this isn't really plagiarism.
You know, there's this one quote that they pullout, um, from the article and they say like,
because they made a copy on their own server,that's prima facie copyright infringement.
That is going to be the claim that Uh, the sort of, uh,the, the newspaper organizations put out the flag, the core
(28:22):
disputes, this comparing PO to a cloud storage service.
So which one is it right in, in practice, it seemslike you really can just use these systems to
like, yeah, get a downloadable PDF to me, naively,you know, from a user experience standpoint.
Yeah, you know, it feels, it feels an awful lotlike, uh, it feels an awful lot like plagiarism.
But, um, you know, they say also, uh, to your point,right, that according to the web server's logs, that,
(28:48):
that, that for the websites that, um, essentially arehosting these news stories, immediately after the assistant
bots were prompted to summarize, Uh, text on the site.
They found a bot identifying itselfas Quorabot that visited the site.
Apparently, as you say, it did notattempt to visit the site's robots.
txt page.
So that robot standard, Andre, that you justmentioned, um, it's kind of worth flagging.
(29:09):
I think we might have talked about it before, but on alot of websites, if you go to like, you know, whatever.
com slash robots.
txt, you'll see there like a list ofbots that are actually excluded by name.
So you'll see like, you know, Quorabot explicitly.
Uh, elicit or whatever.
And that tells you, okay, this page cannot be, uh,cannot be, um, uh, sort of, um, scraped by the core
(29:31):
bot or by whatever the bots are that are there.
Uh, the suggestion here is that essentiallythat protocol is being completely ignored.
And, you know, especially at a time when we'restarting to figure out like, hey, do people by default
have a right to not have their websites scraped?
Um, Or do they have to put it on the robots.
txt?
Or will companies ignore that?
And does the law have to intercede?
(29:53):
Do we need new laws?
Uh, that's a, a kind of interesting exacerbating point.
If you notice in practice, people areignoring these sort of like goodwill, um,
requests implicitly through the, the robots.
txt file.
So, uh, yeah, really interesting.
The core argument here is the file attachments onPoe are created at the direction of users Transcribed
And operate similarly to cloud storage services, readit later services and web clipper products, which
(30:18):
we believe are all consistent with copyright law.
So that's, uh, anyway, a whole bunch of, of interestingstuff there just to say, if you're the one who is
on a one off basis, you know, clicking something andthat's what generates the summary they're claiming.
Oh, well that's, you know, then that's not, um,that's not, uh, a sort of a copyright infringement.
Uh, so anyway, kind of interesting.
I don't know that I buy it personally, but Hey,Again, we got lawyers to listen to the podcast, so
(30:40):
let us know if you think this is a good argument.
Yeah, all these things are so much upin the air, but it does make me think of
these stories about Quora, the perplexity.
It really starts to make sense that OpenAIhas been doing all these media partnerships.
Uh, over the last couple of years, really aggressivelypursuing, paying different outlets millions of dollars
(31:04):
to train on their data, but also to include their datain the outputs to refer to them, cite them, right.
And if it does come out that for paywalled,uh, websites, you cannot read the website to,
uh, generate a response, then this could be oneof the big competitive advantages of open AI.
(31:26):
Yeah.
Right.
As of now, it's a little bit, uh, uh,interchangeable using GPT 4 versus Gemini versus
They're a little bit different, but not that much.
But once these tools can be differentiated by partnerships.
They do become quite a bit different anddepending on your needs, it may make a lot more
(31:48):
sense to use open AI rather than anything else.
So yeah, like developing a sort of paradigm that I don'tthink we really expected or thought about last year.
Yep, and now OpenAI is signing withTime Magazine, so there you go.
Yeah, and on to the lightning round,where we'll try to be a bit quicker.
First, hardware story Huawei and Wushan Xinxin todevelop high bandwidth memory chips amid US restrictions.
(32:18):
This is a report reportedly planning to develop these highbandwidth memory chips, and as we cover, All the time,
uh, China does need some means of producing differenttypes of chips to actually compete in the AI front,
uh, GPUs, but also these high bandwidth memory chips.
(32:41):
And so it makes a lot of sense.
Huawei has been one of the leaders in providing.
AI hardware that NVIDIA is still the leader in.
Now, these companies have denied this report.
They said it's a rumor, but, uh, who knows?
Well, yeah, I, I suspect it's,
I suspect the rumor is true on this one.
(33:02):
But, uh, yeah, they, they've got areally clear reason to deny that.
Um, they're basically fearing more US, uh, sanctions, right?
So Huawei, uh, had the, the hammer come down onthem when they announced to the world that You
know, Hey, we're, we, we partnered basically withSMIC to create a set of sanctions, busting a seven
enemy or technology, um, a couple of months ago.
And then, you know, the U S came down hard.
(33:23):
They're expecting that to happen again, ifthey're not careful about this sort of thing.
Um, one thing just worth, worth flagging.
The one thing I'll say about this is, uh, from a technicalstandpoint, this is about high bandwidth memory, right?
H B M really critical technology distinct.
From, uh, what you might think of as logic, the sort oflike logic chips that are, sorry, the, the logic, uh, on
(33:45):
the chip that, uh, TSMC, for example, really specializes in.
So the logic piece is the piece that runs the calculations.
Um, it's usually really, really close toanyway, this thing called SRAM on the chip,
really, really small amount of memory.
That's just for like working memory.
The high bandwidth memory is what you need inorder to move the massive amounts of data and
(34:05):
all the model weights and the gradients and the.
Activations that you need to run through your systemwhile you're training or doing inference at scale.
So it's more of a kind of pipesthing than a number crunching thing.
Uh, but both are really crucial.
And we talk a lot about TSMC on the podcast.
Uh, they certainly are the leader on the logic side.
When you look at high bandwidth memory, the kind of bigplayers there are more like SK hynix, um, and Samsung.
(34:29):
So each of those, uh, They control about 50percent of the market share in this space.
So this is really China saying, look, we have HuaweiSMIC working together to figure out the logic piece.
And now essentially we've got other playersas well on the high bandwidth memory piece.
And that's really what, uh, is, is up to here.
And, you know, that's going to, that's going to off the U Sgovernment if, if, uh, they make any, any big breakthroughs
(34:53):
here, but it's early days for this category of tech.
They, they very much are behind here.
And next story also on a Chinese company, this timeAlibaba, and the story is that its large language
model has topped global ranking of AI developerplatform, uh, or on developer platform Hugging Face.
(35:13):
So on Hugging Face, that's one of the sourcesfor ranking AI models, and now three of the
top 10 ranked Chinese LLMs of Hugging Face.
Uh, have these Quen series models, Quen 72B, Quen 1.
5, 110B in third and tenth places, respectively.
(35:37):
The other one on there is Yee 1.
5 free 34B chat from the startup, Uh, 01.
ai founded by the notable figure, LeeKai Fu, that was, uh, ranked seventh.
So we see now competition coming around in the spaceof models in China, similar to Anthropic and OpenAI
(35:59):
and Google out in the U S and it's pretty exciting.
pretty clear that as with a lot of technology,there's going to be essentially two
independent ecosystems that will rise up.
And, uh, it's interesting, like there aren't many secretsin the tech involved in making large language models.
(36:20):
So it's basically a race of howmuch compute can you throw at it?
And Alibaba, Uh, Tencent, various companies havea lot of compute similar to Meta and Google.
Yeah.
Yeah.
And you know, the secrets to the extent theyexist, you know, there are some important ones,
but they tend to be at the very, very frontier.
(36:41):
And so they'll take some, some time to diffuse out.
Um, certainly Alibaba is not at the true frontier andlike a lot of Chinese companies, they are incredibly
constrained by the scarcity of advanced technology.
Chips, right?
They don't have access to the best GPUs.
We got these kind of crappy, uh, sort of one and a halfgenerations behind a type chips that we're starting
to see, uh, float around and increasing supply there.
(37:03):
So that's part of the issue here.
It is also part of the reason why somany Chinese companies are focusing.
On the open source angle, right?
There's a lot of reasons why you might want to do that.
You know, one of which is you can imaginewanting to train in certain biases in your open
source models as a way of exerting a kind ofgeopolitical leverage over your adversaries.
Or if you know, American companies, for example, aregoing to use whatever the best open source model is,
(37:26):
Hey, you know, maybe make it talk a little less aboutthe Tiananmen Square Massacre or how bad the CCP might be
treating the Uyghur Muslims or, you know, whatever else.
Um, so this is all kind of partof, of the stuff that's baked in.
The other thing is a kind of legal front that'sbeen open here where we've, you know, talked
about this before, especially with, uh, 0.
1 dot AI, where, you know, their, their terms ofuse say, Hey, if you're going to use our language
(37:50):
model and there's some legal issue that arises fromit, that legal issue is going to be adjudicated.
And the People's Republic of China.
So that's another way that they can get someleverage over what's going on over here.
But certainly Alibaba has made a bigpush in the direction of open sourcing.
Again, like so many other Chinese companies, adisproportionate interest in this because if you're,
you know, you want to be number one at something andthey can't be number one at the true frontier AI game.
(38:14):
Um, then, you know, the, the main thing that that'sleft to you is, yeah, play the open source game.
At least win that for now.
Try to use that to get some headlines, you know,maybe, maybe pull some researchers over to the
extent that you can and, uh, and go from there.
So I think that's kind of where this is all comingfrom that leaderboard, by the way on hugging face.
I just went over yesterday to have a look and, uh,yeah, I mean, it's, you know, it's when number one meta.
(38:37):
So it's like llama three 70 B number two.
And then.
In the number three spot, there's Quenagain, 72 billion with Quen 2, the, uh,
well, it's the, the non instruct version.
Um, anyway, so you see it is quite well populatedby those Chinese models, which are not, not
super surprising, but definitely noteworthy.
Oh, and yeah, worth noting that these are theopenly released models on the Hugging Face platform.
(38:59):
So you're not going to have things like GPT 4 on there.
Next up here comes, uh, meta Ray Banschallenger with chadgpd4o and a camera.
So meta Ray Bans are glasses that comeequipped with speakers and glasses.
And as of this year, meta AI.
So you're able to say, Hey meta, anddo all the usual things you can do.
(39:24):
We have AI, you can say, you know, what is in front of me.
You can ask it questions and it will answer.
And it's sort of been a little underrated or I would say,you know, you've seen things like rabbit are one humane
pin that have been huge flops in the space of wearable AI.
These glasses came out, this is the second generationof these kinds of glasses from Meta and the reviews
(39:53):
are actually pretty positive, partially becausethere's a lot of use for them outside of AI.
You can just record videos from the glasses.
And so now there is this complete solos.
a smart glasses manufacturer, which isaiming to release, uh, similar glasses that
will have QP4O capabilities with a camera.
(40:16):
You can ask it questions.
And evidently you will also be able to integratewith Google Gemini and Anthropix AI models.
So there you go.
It'll be interesting if this turns out to be thewearable paradigm that people actually want, as opposed
to these other, let's say pretty large failures.
(40:37):
Ouch.
Yeah.
Actually, it's funny.
I didn't know not to do real or anything, but I waswatching a YouTube video from this channel coffee Zilla.
And I'm not sure if you follow them, but you've seen it.
Yeah.
Yeah.
So they, they did a thing on the, um, the rabbit R1 and therabbit team basically, and how they were, I had no idea.
They were like a crypto scan basically, as far asyou could tell before they went into the space.
(40:58):
So maybe, um, Yeah, maybe justlike a hype chasing situation.
But yeah, this, uh, this product looks pretty interesting.
It's, um, also it seems to be competitivewith the Ray Bans in terms of pricing.
Maybe, you know, we know that the current version,which does not actually have, um, uh, the video
functionality in it, it's just audio only,uh, is, uh, retailing for basically 250 bucks.
(41:22):
So it's presumably going to be more than that.
Presumably not going to be that much more than that.
So hovering around maybe that 300 pricepoint that the Ray Bans currently sit at.
And I will say just so it's, uh, out of theopen, I am an owner of the Meta Ray Ban glasses.
I'm a bit of an early adopter.
I'm a fan of Meta VR.
I'm a fan of You
(41:42):
don't need to flex.
No need to flex, Andre.
I'm just saying if I'm being very positive aboutthese kinds of glasses, you know why I think.
I do have a personal opinion that this is a sort of wearablePeople may actually want, unlike all these other things,
what do you, what's the mostcommon use that you put them to?
What's the killer feature for you?
(42:03):
The big one is, uh, the video recordingand taking photos just because it means
that you don't need to take out your phone.
And if you're being a tourist or art asan event, that's a pretty nice feature.
You could, you don't need to bedistracted by looking at your phone.
You can just keep looking at whatever you're looking.
(42:24):
This AI integration isn't, I think, yet kind ofa main benefit, but being able to take photos via
voice commands and being able to ask questions, sortof, instead of going to Tragic Betrayal website for
really quick things, you can just use your voice.
I do think, arguably, that could be very useful.
Where is the audio?
(42:46):
you get, you get a response.
No, you get a visual, a video response, right?
And the led not, not an audio response to your, you get
an audio response.
They are built in speakers in the like framesof that use bone conduction technology.
And so you can play music on themto listen to books and so on.
Are they, does the audio go into yourear or do other people hear it as well?
Or how does that work?
It's, uh, yeah, using this.
(43:07):
Fancy technology where it's, you can sortof hear it, uh, but for the most part, yeah.
Oh, interesting.
Okay.
All right.
Well, you, you heard it here at first 300 bucks folks.
Uh, I'm sure Andre, do you have likea, a promo link that you can, uh,
well, you know, unfortunately we're not sponsored by them.
So, uh, too bad, you know, but, uh, anyways,this is a little, just a bit of personal take.
(43:32):
We're not advertising here.
That's great.
And next up, a bit of a spicy story that came out, uh,Apple's Phil Schiller is reportedly joining OpenAI's board.
This will be, uh, an appointment with an observationcomponent, so not kind of directing OpenAI's board.
(43:53):
Actions, and this person is the app storechief and the former marketing head.
So they go, you know, the deeper partnership betweenApple and opening eye continues to get deeper.
It does.
The other thing, too, that's sort of interestingabout this and that jumped out at me when I saw
(44:15):
this was, uh, boy, this is going to be awkward.
You've got the Microsoft Board Observer on the board.
You've now got the Apple Board Observer, like theMicrosoft Apple rivalry, two people on the same board.
By the way, uh, Apple not known.
For taking board seats on prominent companies very often.
They have done it a couple times.
Um, I think there's a Chinese guy who's, I think itmight've been DD that they, uh, they, they did that
(44:39):
briefly on, but not a standard practice for Apple at all.
This will give them some visibility into the machinationsof what's going on at the board board level and, and
how are, how's company business being handled basically.
So, you know, that's pretty strategic information.
It's just about all they get from it because itis, you know, just like Microsoft observer role.
Um, but, you know, some intel it's going to comeout of that you could imagine situations where the
(45:02):
Microsoft observer might request that, uh, thatthe Apple observer leave, for example, if they're
going to talk about the sort of Microsoft opening.
I partnership in a way that strategic.
Those sorts of requests, as I understand it, are oftengranted when they are made, but that is a courtesy.
Um, so, anyway, kind of interesting, uh,kind of interesting times over at OpenAI.
(45:22):
It just reminds you of how insane it is that Sam Altman'sbeen able to make these deals between Microsoft and Apple.
I mean, this is like the original OG tech rivalry.
So, uh, so there you go.
Yeah, it really makes me wonder if in some sense, Microsoftisn't entirely opposed to this in a sense that we have
talked about how there were antitrust investigationson Microsoft and OpenAI in Europe and in the US.
(45:46):
So similar to how in the nineties, Microsoftkind of propped up Apple in a kind of funny way
and made antitrust considerations against them.
Well, maybe they do want OpenAIto partner with other companies.
On the other hand, it does make me wonder, uh,Microsoft apparently paid and invested in OpenAI for
(46:07):
this exclusive license to GPT technologies, which wasalways pretty vague and unclear on what that means.
While the exclusivity thereapparently is not, uh, holding up.
So, very curious situation.
So
that, okay, that's a great point, number one.
Number two, something it also brings to mind, right,is OpenAI's board is, so, so the voting part of
(46:30):
the board, not the observers, but the voting partof the board, is charged with determining when
officially OpenAI can be said to have achieved AGI.
That matters because when they do, thatrequirement on the part of OpenAI to share
their technology with Microsoft doesn't matter.
And OpenAI is not required to shareaccess to technology that's AGI and above.
(46:50):
And so, you know, you can imagine if you have Apple now,even as an observer, maybe influencing conversations on
the board, trying to steer things towards say, recognizingearlier on that they have hit AGI or some similar threshold.
That could be an interesting way to throw awrench in the, in the, in the gears anyway.
So we'll have to see.
No, you're totally right.
This is such a, it's such a weird situation.
(47:12):
We're just completely off the edge of the map here.
So, uh, we'll see what Phil Schiller does.
Yeah.
You know, if, if there are any antitrustexperts, uh, you feel free to chime in.
I think it's very interesting to thinkon whatever that might be a possibility.
And the last story for AI video startuprunway is looking to raise 450 million.
(47:34):
dollars and that would put the valuationof a company at four billion dollars.
This is following up on previous raising in 2023.
They raised 141 million from investors suchas Google and NVIDIA, and we're valued at 1.
(47:55):
5 billion.
So yeah, you could say that things are heating upquite a bit for Runway and, uh, we'll be interested
to see if they actually do raise that amount andinclude increase their valuation by that much.
Onto projects and open source.
First section, Qtai open sources Moshi, areal time and native multimodal foundation
(48:19):
AI model that can listen and speak.
So this was a pretty big thing on Twitter, Ithink, where this model Moshi is pretty much
what GPT 4 0 demonstrated, a real time multimodalmodel that you can talk to in near real time.
The model was fine tuned with 100, 000aural style synthetic conversations and has
(48:45):
an end to end latency of 200 milliseconds.
And so yeah, people are prettyexcited to have a GPT 4 O type model.
I will say that looking outside the demo atsome videos posted on Twitter, people trying it
out, not very impressive, but still people aredefinitely working on these kinds of models.
(49:06):
Yeah, I feel like that's happening an awful lot these days.
We're definitely, we've crossed the uncanny valleyin demo land, but beyond that, it sort of like
reminds me of, um, I'm trying to remember whatthe name of that company was, but start with a V.
Anyway, um, you know, back in the day there waslike this, uh, how this is going to crush me.
Anyway, there's, there was some company thatwas really good at doing demos of AI stuff and
(49:27):
it just never, never came, uh, came together.
And now it's really bothering me.
It doesn't matter.
Doesn't matter.
I'm not bothered by this at all.
Um, so this is a, yeah, 7 billion parametermodel, uh, which, you know, surprisingly
small, especially for a multimodal model.
You know, that's, that's probably wherethey're getting the low latency from too.
It's not that much to kind ofpropagate through it inference time.
Um, but if it's also, you know, not delivering in quality,then that's, you know, maybe a more fundamental issue,
(49:51):
but, uh, yeah, the diffusion of the stuff recedes a pace.
Look, if this model actually worked.
Um, you'd have some really interestingquestions about, uh, responsible use, right?
You got a system here that can do the textto speech stuff, can do all the GPD 4.
0 stuff, again, if it actually could, whichit kind of can, but really kind of can't.
Um, and, uh, you know, we ought to be askingourselves, like, what, what does this imply about,
(50:15):
you know, scaled, uh, not just disinformation,but identity theft and all the usual things.
If, you know, phone calls, AI generated phone calls, orjust an API, uh, call away, then that's, uh, you know.
Probably a bit of a shift interms of that risk landscape too.
But, uh, but anyway, I'm not at all botheredby the fact that I can't remember the name
of that company that starts with a V now.
And, uh, and that's
all I got.
That's a good thing.
(50:36):
I guess, uh, sometimes you got to just let things go.
Yeah.
Yeah.
Like that company whose name starts with a V.
Anyway, and next we have a paper mm eval procalibrating multi modal benchmarks towards
cross theory and efficient evaluation.
This is a partnership of a few organizations and theyshow that on multi modal benchmarks with multiple choice.
(51:05):
questions.
Often, the numbers are a little bit wrong.
So you can use biases in the questions to answer or guessthe answer pretty well without even using the image.
There are sorts of things that happen often with benchmarks.
You know, you realize after release that theway the models are getting good performance
(51:27):
are not really the way you want them to.
So in this case, Updated benchmark, there are2, 138 question triplets, uh, two thirds of
those were manually labeled by human experts.
And some of them, uh, some of the otherones are from existing benchmarks.
And as a result, this benchmark is more challenging with thebest LLMs lagging behind human performance by 31 percent.
(51:56):
percent compared to eight percent in previous benchmarks.
And that's, I think what more and more of thesenew benchmarks are seeking, you know, actual
challenge where VLLMs aren't at human level.
Yeah, it's, it's so interesting, you know, not atall, frankly, how I would have expected the space to
play out like, you know, 10 years ago, five years ago.
Um, you know, you're looking at it now.
(52:16):
It's like the, we seem to be smashing, um, you know, humanlevel performance across such a wide range of, of tasks.
And then the ones that were not, um, so often it kindof feels like this Uh, you know, this famous like good
hearts law problem where the minute that you identify ametric and make it a target that you're going to try to
optimize for all of a sudden, yeah, you'll find ways tomove that, that metric that don't actually touch the kind
(52:39):
of original underlying cause of that metric being so low.
So you know, you can kind of overfit, let's say to aparticular performance metric and uh, and not actually
address the substance of what it used to be getting at.
You can hack that particular metric.
So you know, this is kind of highlighting that that isabsolutely an issue, especially in the multimodal domain.
Maybe a bit less surprisingbecause that domain is a bit newer.
(53:00):
Um, but yeah, their big focus here is on the sort of liketype one error, uh, to kind of abuse my statistic language,
a bit of a, like kind of false positive type error where youincorrectly conclude that, yeah, your model's really good.
Um, when in fact, what's going on is it's, youknow, using cues in the question that kind of
don't require you to look at the actual multimodalpart of the problem at the image, for example.
(53:24):
Um, you know, it's, it's the.
If just the way that from the way the questions asked,there's a little bit of information that makes you a
little bit more likely to answer the question withouteven needing to look at the, um, at the image that it's
supposed to be coupled to, then that can be a thing.
And I think that's going to be an ongoing challengehere, but, uh, but yeah, another, a new benchmark
that shows surprising, uh, uh, The surprisinggap between humans and AI systems is always good.
(53:45):
Uh, it's an area for, for improvement for sure.
Next story also on model evaluation.
And this one is about how Anthropic ispushing for third party AI model evaluations.
So they had a blog post titled a new initiativefor developing third party model evaluations
where they asked three key areas of focus forevaluation development, AI safety level assessments.
(54:12):
Advanced capability and safety metrics and infrastructures,tools, and methods for developing evaluations.
And so the third party aspect of thisis that we are seeking proposals.
They have an application form for, I suppose,anyone to, uh, submit a proposal for evaluation.
They also have a new email address.
(54:35):
If you have questions, you canemail eval initiative at anthropic.
org.
Calm.
So, uh, yeah, it makes a lot of sense.
I think this is part of the conversations and safetyneeding, uh, independent, uh, overview kind of initiative.
And this seems to be in mind of that.
Yeah.
I was, um, at some conference a couple ofweeks ago with, uh, I think, you know, Jack
(54:58):
Clark gave some, some, um, address about this.
And this is basically core to anthropics.
Now, big governance play.
So one of the things that they're imagining is this futurewhere you have these things called regulatory markets,
where basically you have the big companies building outtheir AI systems that could potentially be dangerous.
And then you have this ecosystem of smaller companiesthat are tasked with performing audits and other
(55:23):
things like that for those bigger companies.
And those smaller companies.
Anthropic one day hopes, or at least Jack Clark oneday hopes will have some sort of government mandate
or support they'll be, you know, getting, gettingcertification from those private companies will be
required in order for those, those big companies torelease models and maybe do some development work as well.
Um, you know, I, I think they're.
(55:44):
Some, some significant issues with this, uh, can be in termsof how far you could go with things like misalignment risk
with things like, um, C burn, like chem, bio, radiologicaland nuclear risks, just because, you know, you need so many
companies to be performing these audits, uh, with clearancesand, and, you know, accessing very, very sensitive
data, but you know, doesn't, doesn't mean it can't work.
(56:06):
Um, and this is certainly them firing the, the initialshot here, really trying to drum up this ecosystem.
They've done a great job of that.
They've.
You know, brought on a whole bunch of, uh, great AI,uh, evals companies to audit their models in the past.
So they're really trying to create, stimulatethat demand at the, at the ground floor, um, which
you know, is a, it's just a good thing, right?
You want more companies that can do more of these audits.
(56:28):
Um, I will say one thing.
Looking at their AI safety level assessments section, right?
This is the thing that, you know, one of the three keyareas that they, they want to source, uh, contractors
for, it was kind of interesting to see them break downthe, the, the risks essentially that they want to look at
their, look at their models for, you know, they call out.
Your standard, you know, cyber attack, cyber offenserisk, chem, bio, radiological, and nuclear risk.
(56:51):
This is sometimes known as C burn risk.
So we've seen that call that before autonomy as well.
These are like, you know, can the models selfreplicate or, uh, you know, gather resources
on its own, the so called survival andflourishing evals that we've talked about before.
And, uh, and then there's social manipulationand then there's misalignment risk.
And they, They kind of have misalignment riskas its own category, independent of autonomy,
(57:15):
which I think is both interesting and important.
Uh, that's I think the right call, but kindof interesting that they're separating, right?
The ability to self replicate to kind of escape if it,you know, if it wanted to from the actual, uh, impetus,
the desire to do that or the incentive to do that.
So, uh, yeah, I thought it was areally, uh, interesting breakdown.
Anthropic is, is good at this.
They do a good job of kind of layingout there in a very structured way.
(57:38):
They're thinking on a lot of these risks sets.
So, uh, yeah, great, uh, great read andwe'll, we'll see what comes out of this.
Yeah, hopefully what comes out is no AI doomsday.
But then what would we talk about Andre?
And one last story for thissection is about Mozilla Llamafile.
And this is coming out of the AI EngineersWorld's Fair, which just happened in the Bay Area.
(58:06):
So the announcement of this Llamafile, if thisis an open source project and Llamafile's are
these files that package together the weightsof an LLM with the software needed to run it.
With the idea being that with these files, it'll be mucheasier to deploy models on device and in various platforms.
(58:30):
Mozilla, of course, is a developer of Firefox.
They are pretty big in the open source space.
So this is an open source initiative.
to power local AI and Mozilla also unveiled thisMozilla builders project, which is an accelerator
that offers 100, 000 in funding for open sourceprojects that advance promise potential of local AI.
(58:55):
So many companies are trying to become a player in the opensource space and interesting contribution here from Mozilla.
It's also, you know, Mozilla, obviously abig contributor to the open source game.
Uh, that's really become their big differentiator,especially in the last couple of years.
Um, but they've, you know, they've always been into itand, you know, Hey, the Firefox browser, there you go.
(59:16):
Uh, big open source contribution.
So now more and more so on the AI side.
Um, one of the, uh, the interesting things about this too,is that llama files are actually going to intelligently
utilize, as they put it, they could have said use.
I could have said you, just, just saying,but they, they utilize, they're going to
intelligently utilize available hardware.
Um, so for example, they can use a GPU if availablefor faster performance or your CPU if it's not ready.
(59:40):
So that's kind of interesting that they've done a lotof like optimization and, and there's a bit of meta
awareness of the, the hardware ecosystem around this thing.
It's a single executable file.
So just basically, you know, double clickif you will, and, and, uh, start it running.
So yeah, really cool and consistentwith their, uh, their wider approach.
Thank you.
Onto research and advancements.
First up, we have researchers append AI statusquo by eliminating matrix multiplication in LLMs.
(01:00:07):
So matrix multiplication is basically mostof what neural nets and large networks do.
do they take some weights, some inputs, multiplythem a bunch of times, and that produces the outputs.
So this coming from university is offering a wayto, uh, get around that by using ternary values, a
(01:00:30):
fancy term for just using negative one zero and one.
instead of floating point numbers, things like 1.
1, 1.
2, and so on.
And they have implemented this matmul freelinear gated recurrent units that can perform
basic arithmetic operations, and so on.
(01:00:53):
So with the approach, the big benefit is, of course,reducing memory usage and potentially also increasing.
And, uh, they do have some experimentson scaling up models up to 2.
7 billion parameters showing that they arecomparable in performance while using much
(01:01:16):
less memory and having better latency.
Yet another example we've seen.
Uh, one bit models also, uh, published,uh, this appears to be quite similar.
And so we have to wonder, you've seen a coupleof papers generate some discussion, uh, but, uh,
it still has yet to be proven at larger scales.
(01:01:39):
I think that's, so that's exactly right.
Especially on the sort of speculative nature of this.
It is exciting.
Um, so just for, for context, essentially what'shappening here is instead of doing matrix multiplication
algebra, which is, What usually has to go into makinga transformer do what it does, what they're doing is
something that that is something like addition, thatthey're kind of swapping out multiplication for addition.
(01:02:01):
Intuitively, you can see how that would work, right?
Like when you're just dealing with threepossible values, negative one, zero and one.
Um, the, the kind of, um, operationspace is a lot more constrained.
And so multiplication and addition, uh, startto kind of allow you to reach just about as
far in, in, in that, that kind of value space.
(01:02:23):
Anyway, bottom line is addition is also a lot easier, right?
It's a lot faster to do.
And so by stripping out the The multiplication side, you'reable to kind of run these things much more efficiently.
And they actually do this on their own FPGA,basically like custom hardware, because
ultimately everything is so optimized.
All the hardware that exists is so optimized fortransformer architectures that you can't really, like,
(01:02:46):
if you come out with a new crazy good idea for, youknow, these kinds of architectures, You're probably
going to find that it underperforms a lot just becauseit's so, um, handicapped basically by the fact that
hardware is so optimized for the current paradigm.
Um, one, one little note here, and there was a blogpost, uh, that I was looking at on, um, I think it
(01:03:07):
was less wrong that went into this a little bit.
I just looked up the title and this thing popped up.
So they highlight a couple ofimportant caveats to this, right?
So one of them is.
The, the curves that they use.
So, so just, uh, maybe for a little bit of context,the key thing that they're doing here is they're
demonstrating that the scaling curves that you getfor this architecture actually seem to look better
(01:03:32):
than the scaling curves that you get for transformers.
So traditional transformers, when we talk aboutscaling curves, basically we're talking about how.
Low is the loss.
How well does the model perform as you pour more computeinto it, as you put more flops into the training process?
And what they're showing here is that that curve forthis, uh, sort of, um, uh, addition strategy instead
(01:03:52):
of multiplication, the addition, um, architecturethey're using here, that curve is actually steeper.
So as you increase the amount of flops that you pouredyour model, supposedly you're, you're getting, um,
kind of more, more returns that would be a giant deal.
And these These curves actually intersect at 10to the 23 flops or so, which is actually the scale
(01:04:14):
of a lot of open, uh, current open source models.
It's not a huge, huge scale.
So the argument here would be, it's alreadybetter to use this kind of architecture,
assuming that the hardware is there for it.
Uh, the challenge though, is that they're drawingthat conclusion based on three points on this curve.
They're really extrapolating from three points.
Given how these points look, I'mjust looking at the raw data here.
(01:04:37):
You could very credibly make the argumentthat this is actually, it's a, yeah,
it's, it could just be noise basically.
So you, you would, to your point, Andre, youwould want to see this done at more scale.
You just want to see more points on that scaling curveplotted out, do more training runs and, uh, and verify that
this is actually the, the trend that you expect to hold.
Cause you know, right now this could just, It couldbe, frankly, a bit of a nothing burger if it just
(01:04:59):
turns out that, hey, uh, when you actually redoit, you find that the traditional transformer, uh,
either matches or, or outperforms this architecture.
But, but still, I think it's, it's interesting andnoteworthy that such a simple architecture that just
does addition, like this fundamental assumption thatI think it's fair to say we all would have made that
multiplication was pretty core to what transformersare up to and the magic that happens under the hood.
(01:05:21):
That, doesn't seem to be the case.
So, you know, it makes you wonder whatother low hanging fruit might there be.
And, and that's probably a pretty bigfield of possibilities at this point.
Exactly.
And we've seen this a lot with quantization, right?
Where it's pretty well known nowthat you don't need all the numbers.
You can reduce it to three bits, four bits, et cetera.
(01:05:42):
Let's see if we can take it down to, uh,what, two bits, I guess, in this case.
I'm just happy that they didn't callthe paper addition is all you need.
Cause that would have
Yeah, that'd be a little, uh, obnoxious.
And speaking of a popular paper name, uh, formulas, thenext one is AI agents that matter, uh, with, I've seen,
(01:06:06):
uh, maybe one or two times a few years ago, there was deepreinforcement learning that matters, that called out the
evaluation practices of reinforcement learning and basicallypointed out that they suck, that they don't really compare
things well because you know, you don't control for thingslike seed values and randomness and, uh, things like that.
(01:06:29):
That's essentially what this paperis doing for agent evaluation.
And the big thing they point out is that onevaluation, you cannot just report accuracy.
You need to, uh, Produce the accuracy whileaccounting for cost in the sense of that with
agents, if you just run more of them or run themfor longer, you are likely to get better results.
(01:06:54):
So we need standardized benchmarking practicesthat allow for actual information, reproducibility.
And, uh, there's a few more other things that,uh, they do analysis of failure modes, for
instance, overfitting on a single task, meaningthat they won't generalize, uh, things like that.
(01:07:18):
So I think a very good point, uh, for the space, uh, agentsare kind of, you could say the next frontier that are seeing
a lot of, uh, work and a lot of, uh, investment and so on.
We are not quite there yet, but A lot of peoplethink that this is the next thing to crack.
(01:07:38):
And this is one of those things that showcase that weare still a little bit early on on the road to that.
Yeah.
I really liked this paper.
It was, it was a great example of kind of questioningsome of the, the basic assumptions that nobody,
you're always surprised when you see papers like this.
Cause you're like, wait, nobody, nobody asked thisquestion, but then again, it's not like I did.
So, so there you go.
Um, yeah, no, they, they do a great job.
(01:07:59):
They, they set up a couple of, differentbaselines that they compare to each other.
Um, so for example, they, they show youlike, okay, GPD four models, just like out
of the box, zero shot, no agent architecture.
How do they perform?
Um, then they have a version they call retrywhere basically you just repeatedly call.
The model, um, up to, up to five times in this case and tryto get it to solve a problem and see how well does that do?
(01:08:26):
Um, then they have a version that's kind of similar.
They call it warming same idea, but you just turnup the, the temperature of the model with each run.
The temperature is this parameter that kindof controls the creativity of the model.
So how often will it predict, uh, will it generatethe word that is maybe not the most likely next
word it thinks are the most optimal next word.
(01:08:46):
But, you know, one of the kind of from the wings thedistribution a little bit more swing for the fences.
Um, so that's that warming example.
And then there's this thing called escalationwhere they start with a cheap model, uh, in
this case, llama three, 8 billion parameters,and they escalate to a more expensive one.
So say GPT 3.
5 or something like that, if theyencounter a test case failure.
So those are the, the sort of four paradigms,pretty basic approaches, not really your,
(01:09:11):
your agent like models that you would expect.
And they draw a couple of really,to me, interesting conclusions.
Um, You know, one of them is that state of the artagent architectures, at least for a human eval,
they don't outperform these simple baselines.
They find there's no significant accuracy differencebetween their, in this case, their worming strategy.
(01:09:32):
So pretty out of the box, uh, kind ofbenchmark baseline and the best performing
agent architecture, at least for this eval.
So human eval is a, essentially a coding datasetwhere you have Um, kind of coding tests that the model
has to pass and um, yeah, like there's no measurableincrease in performance with these fancy architectures.
Furthermore, a lot of these fancyarchitectures suck in terms of costs.
(01:09:57):
So not only are you not getting better performancerelative to these, you know, fairly vanilla, you know,
Um, essentially like prompting engineering strategies,but you're also getting outrageously higher costs.
They find that techniques like reflectionand this thing called, uh, LMDBs,
sorry, LMB.
So this is the language model debugger.
They cost 50 percent more thanthe just vanilla warming strategy.
(01:10:21):
And another one called language agent tree search,which we've podcast before cost over 50 times more.
Well, yielding basically the same results.
So then they, they end up showing this curve basicallyof the cost of these agents on, on the, the X axis
and the accuracy, basically the performance onthe Y axis, and they trace out the sort of, uh,
(01:10:42):
well, the Pareto frontiers, it's called, right?
So this is like the frontier of what's optimal.
And from there, you can really see how some of thesearchitectures just really fail to, to perform as well as.
You know, the escalation or the warming, um,baselines, which are again, like not fancy.
They're, they're really basic.
And so this really calls into question, um, they'regoing to argue, uh, whether it even makes sense
(01:11:06):
to try with these very expensive, very complexarchitectures when, you know, pretty simple prompting
strategies, uh, naive strategies do the trick.
Yeah, and that is something we've seen inmany cases, or at least I've seen over the
years with deep learning and machine learning.
Every once in a while you have this kind ofpaper that comes out and for a given domain,
(01:11:27):
in this case agents, but it's also happened forvideo evaluation, for question answering in NLP.
A lot of different things, it basically says,look, it turns out that these numbers are
mainly due to how we are doing the benchmarking.
Maybe the benchmark itself, uh, for instance, like you canguess the result about looking at the video or the image.
(01:11:50):
And then they say, well, here's a very simple thing thatturns out to work way better than all these other things.
And so, yeah, very much a useful thing.
that comes out every once in a while to point out sortof the mythological flaws of the community as a whole.
At the lighting round, we start with a bit of a nerdy paper.
(01:12:14):
This is such a nerdy paper.
It's really quite nerdy, I would say,but we are going to cover it nonetheless.
It's called the Warp on the Benefitsof Weight Average Rewarded Policies.
So if I understand correctly, it's tackling this problem.
of when you do a reinforcement learning humanfeedback on models to align them with what
(01:12:38):
humans do, you're starting with a trained modelalready with a bunch of knowledge baked into it.
And so if you train it some more to make itdo certain things that you want it to do, As
a result, it may, uh, lose some of that stuffthat it had prior to this additional training.
It can forget pre trained knowledge.
(01:12:59):
And so usually when you do this stuff, thereis a technique called KL regularization, which
essentially makes it so the models stick.
pretty close to what they were prior to the RL,and this paper has this weight average rewarded
policies strategy that shows that you can get awaya little bit from, uh, vanilla KL organization.
(01:13:27):
You can do some fancy stuff to, uh, improve the ability tooptimize in the RL stage while still retaining information.
Yeah, exactly.
That's it.
So this, this is basically from the question,the problem of like, if you, so in reinforcement
learning from human feedback, usually what you dois you create this thing called a reward model that
(01:13:47):
you train from a lot of interactions with humans, alot of upvotes and downvotes of different content.
That reward model is naively, you would think, okay, well,I can just train my language model to get a high reward.
high predicted reward, uh, to generate outputs.
Let's say that that reward model would predict would score.
Well, um, the problem is that you end up with these perverseoptimizations where the model will then find ways to make
(01:14:12):
that reward model, get a really high reward or offer areally high reward, but that are basically just hacks.
Like it finds the, the.
Through the breaking points, the ways in which that rewardmodel maybe doesn't fully capture what you want, um,
because how could it human desires and values so complex.
And so, yeah, what you do is you say, okay, welllook, head a little bit in the direction of what that
(01:14:33):
reward model wants, but don't change too much, right?
We want to make sure that, you know, you, youdon't, um, you don't change your behavior too much.
And so you include this, essentially this anchor,uh, as it's sometimes referred to this, uh, callback
labeler, uh, divergence, Um, kind of score that says,hold on, you're, you're starting to behave a little
too differently from your original behavior, solet's kind of not go off too far in that direction.
(01:14:54):
And the perverse optimization doesn't happen.
This is a technique that is going to say, okay,for any given fine tuning run like that, um,
what we'll find is if we take the same, if we,let's say, make five different copies of our
language model and we run it through this process.
Five independent times where, you know, we'll, we'lltrain it on our reward model and with that, you know,
(01:15:17):
whatever that callback label, label, labeler thing,so it doesn't go off into some random direction.
We'll do the same thing with the second copy and third copy.
These copies are going to learn to get highscores in slightly different ways, right?
They'll learn to get high scores for that rewardmodel in slightly different, different ways.
And so they'll all learn kind of slightlydifferent things about what humans do.
(01:15:38):
actually want that was encoded in that reward model.
And so what you then do is you train a bunch of thesein that way, and you're going to merge them together
by, actually it's a dead simple technique, basicallyit's, it's like averaging their, their weights together.
The, the parameter values, um, it's a little bit morefancy than that, but anyway, that's roughly the idea.
Um, and then they are going to essentially, There's a fancyway of doing this, but roughly they take that, uh, that
(01:16:02):
new model they get and they're going to repeat the process.
Okay.
Use that as the new language model, create fivecopies of it again, run through this process where
each copy gets trained independently against thereward model and the callback labeler divergence.
It doesn't go too crazy and then merge them again.
And so they go through these kinds of independentcycles, uh, to gradually get a more and more Aligned
(01:16:26):
model, but that doesn't go off the rails, and theresults are, um, anyway, are pretty interesting.
It's actually, again, one of these Pareto frontier things,because by controlling the number of times they go through
that loop, they can more finely control how much do youorient the model towards what the reward model wants.
Towards your, your assessment of human preferencesversus how much do you get it to remain as it was to
(01:16:48):
kind of, you know, orient towards that COBAL labelerscore so that it doesn't change too much from what it
originally contained the knowledge it originally contained.
Um, anyway, so that's, that's basically the overview.
It's super nerdy paper.
Sounds super nerdy, but also worth noting.
This is coming from.
DeepMind.
And so while it seems theoretical, it could wellhave significant impacts in practice, since of
(01:17:11):
course, the stage of, uh, RL from human feedbackis essential to developing a good chatbot.
Next up, scaling synthetic datacreation with 1 billion personas.
The offers here introduce this thing called Persona Hub,which is a collection of 1 billion diverse personas of LLMs.
(01:17:36):
Uh, they QA'd this from web data and these are meant tobe sort of different, uh, mimicking of different possible.
Ways of thinking of knowledge, uh, these kinds of things.
And the purpose of it is to do this synthetic datageneration, uh, generating data that doesn't exist via AI.
(01:18:03):
And then being able to use that for, uh, trading models.
So, uh, yeah, I like this paperbecause 1 billion is a big number.
Personas.
Sounds cool.
And persona hub sounds cool.
Uh, and also because synthetic data is one of thesethings that might be essential moving forward, it is
(01:18:24):
probably already something that when you look at GPT4 or Uh, Claude, that is part of a training recipe.
And so this thing of, uh, having, uh, differentvariations of your AI that ensures that the
synthetic data is actually useful, that producesvery data, this could be an approach towards that.
(01:18:48):
All right, cool.
So, so we know that one, one billionis Andre's big number threshold.
That's good to know.
Um, so this is, yeah, this is actually a reallyinteresting paper as well, because I think
it highlights how counter intuitively hardit is to generate a bunch of synthetic data.
Like this is not something that, you know, I think mostpeople would think, well, you know, you got GPT 4, right?
Just, Ask it a bunch of questions andyou'll get your, you'll get your data out.
(01:19:12):
But of course you're limited then by how manydifferent ways you can phrase the question.
Uh, because if you ask the same question, you know,identically twice, you're going to get the same answer.
If the temperature is down to zero modulo,uh, some reasons why you actually wouldn't,
but that are not terribly relevant here.
Um, so this is a bit of an issue.
The solution that they came upwith was like, Hey, wait a minute.
(01:19:33):
Why don't we just, um, if we can create a bunchof descriptions of personas, a gigantic number,
let's say a billion, so it'll impress Andre, um,then we can actually take those descriptions of
personas and append them to a bunch of questions.
So, you know, for, think about like,um, okay, explain how a rocket works.
(01:19:54):
Okay.
You know, you might get one output for that.
Okay.
Now explain how a rocket works for a, um, you know,a software engineer, mother of four in Oklahoma.
Okay.
Now explain how a rocket worksfor a doctor in blah, blah, blah.
Right?
So depending on the persona that you tailor itto, you're going to get a different output that.
Okay.
Kind of shares, maybe a different aspect of theknowledge that was encoded in the module in the
(01:20:17):
model that you're prompting, and that's important,because really, when we're talking about generating
synthetic data, we're talking about trying to elicitas much as possible all of the information that is
contained Latently in the model that we're prompting.
And the only way to do that is to poke it from everypossible direction we could possibly imagine poking it from.
And personas are identified here as the way to do that.
(01:20:39):
And the big question is how do youactually generate the personas?
And essentially the, the technique they're goingto use here, the text to persona strategy that
we'll use is to give a model, a piece of text.
that they pull from the internet and ask the modelto describe the kind of person who would read,
write, like, dislike, whatever that text, right?
(01:20:59):
So you're going to get from that a descriptionof a persona and you can get a giant number
of them because you're querying a model.
And then you can also, so one of the challenges thatyou run into with that is you might not actually
find that all personas that are possible are actuallyrepresented there because the text on the internet is
disproportionately written by certain kinds of people.
And so they add onto that a persona to personastage where they derive personas with inter based
(01:21:25):
on the kind of interpersonal relationships theywould expect to exist from the first personas.
So basically figure out like who are the kindsof people who would work with or interact with
the personas that you previously identified.
So then you end up with this gigantic set of personas.
You trim them down using various techniques tomake sure that, you know, cause you'll have a
ton of overlap, a ton of redundant personas.
(01:21:46):
Um, they do that, they end up with over a billion personas.
Worth flagging, this is a paper by Tencent AI Lab Seattle,that is a Chinese company for all intents and purposes.
And so, um, it notable that they write, uh, we areopen, sorry, we are open to releasing more data.
They release, I think, uh, I think it was about,uh, 200, 000 of these personas openly, but they
say we are open to releasing more data when we canbetter assess the potential risks and concerns,
(01:22:10):
which will be discussed in later in detail.
Um, this is a, it's basically a CCP lab, but okay.
Um, at least I should say Tencent is TencentAI lab, Seattle, uh, you know, may have a,
let's say a complicated relationship withTencent, but it, it certainly makes you think.
Yeah, and, uh, some examples of personas, uh, anenthusiastic amateur golfer who is passionate about
(01:22:32):
combining sports with philanthropic causes, uh,and also a senior software engineer who encourages
the undergraduate to consider the social impact oftheir algorithms, uh, some of the shorter ones they
have in the first page, a chemical kinetics program.
researcher, a moving company driver.
(01:22:52):
So that gives you some of a taste.
Uh, there you go, that they use those personas to createa bunch of data, uh, for different, uh, interactions.
And yeah, there you go.
I think a bit of a less nerdy paper because,uh, personas are fun to think about.
And, and there is, there is actually kind of a big headlinerhere, which they kind of buried this lead a bit, but they
(01:23:14):
do this with a, like an open source 7 billion parameter LLM.
It's a Chinese model.
It's quen two, right?
Seven B.
Um, they actually, so they run this hard.
They do the, um, they're sort of finetuning on synthetic data that they generate.
That's kind of like math related.
And they actually end up with a performance onthe math benchmark, uh, that beats Claude three.
(01:23:38):
Opus.
So like, I don't know, a couple of weeks ago, thiswas the leading, um, the leading anthropic model.
This is the big model, right?
It's, it's like a Gemini ultra.
It's the largest scale version of the quad three family.
It beats this model.
It's a 7 billion parameter model.
It beats that model.
Um, uh, it basically on NGPD forturbo, by the way, the preview version.
(01:23:59):
inaccuracy on the math benchmark.
So, you know, looking at one particular benchmark, notalways, uh, doesn't always give you the complete picture,
but it is noteworthy that when it comes to just, youcan do this by cranking out synthetic data at scale.
If you have these personas, this technique, at leastso far seems to be real, seems to work quite well.
And one last paper found in the middle, CalibratingPositional Attention Bias Improves Long Context Utilization.
(01:24:25):
This will be the a little bit mid dirty paper.
Mid dirty, okay.
Yeah, yeah.
So, uh, they look at the problem of when you havelong inputs to your model, There is this problem of
if the thing you care about is in the middle, then,uh, LLMs typically are not as good at using it.
(01:24:50):
They're better at looking at thebeginning and the end of the input.
And it seems like this is a problemknown as the lost in the middle problem.
And one big reason for it is because of anintrinsic attention bias where tokens at the
beginning and end of input receive higher attentionregardless of the relevance just due to how you
(01:25:14):
compute attention and positional embeddings.
So to fix that, the researchers propose a calibrationmechanism found in the middle that fixes that
bias and as a result, uh, makes it so you can dealwith stuff that is in the middle of your prompt.
Yeah.
And then the fixes, the calibrationis surprisingly straightforward.
(01:25:36):
Uh, roughly speaking, what they end up doing is comingup with like a, a dummy, um, like a, yeah, dummy document
that they're going So put at various positions in this,um, in the context window and see what is the attention
score on that dummy document, uh, on average, right?
The, the average attention scoreacross all the tokens in that document.
(01:25:56):
And that's going to allow them to say, okay,well, you know, um, if it's an irrelevant dummy
document, uh, that variation, we can just add thatattention score rather, sorry, we can subtract
that attention score, um, from whatever we get.
from our actual, uh, kind of documentat that position in the context window.
And that should give us the kind of calibratedversion that doesn't have this positional bias.
(01:26:16):
So again, like one of these cases where it's, it'sso simple, you find yourself reading these papers
and it's like, how has nobody thought of this?
But of course there's just so much going on.
And the reality is a lot of these techniques don't work.
Don't need to be optimized much to work, um, be itbecause they're, you know, operating at crazy scales or
everything else is already so optimized or whatever else.
So yeah.
And interesting that we're still learning such fundamentalfacts about how the, uh, the, you know, kind of
(01:26:41):
context window works and attention values are computed.
And moving on to policy and safety, the first storyhighlights, uh, has a bunch of stuff in the link,
but, uh, one main thing, which is with Chevron'sdemise, AI regulation seems dead in the water.
So this is in the US.
Recently, the Supreme Court has struck down theChevron deference, which is a policy that allowed
(01:27:08):
federal agencies to interpret congressional laws.
And that is now struck down, meaning thatnationwide AI regulation and generally a lot
of policies coming from the executive branchof a government, uh, are less, uh, powerful.
And this story comments on that.
(01:27:31):
That means that courts will have toexercise their own legal judgment.
And, uh, regulation will need to go through, uh,be able to survive specific legal challenges.
And given that these are pretty technicalsubjects, uh, we'll See if the courts are able to
(01:27:53):
deal with this chevron deferenceis a it's a really big deal.
Actually, it sounds it sounds boring It'sactually not it's kind of an interesting concept.
But but it's super super important.
I mean, this is
One of the most important probably the most importantpolicy story of the last year And and maybe at the next
year too, depending on how things go in Congress, butessentially yeah the idea here is Any time right now,
(01:28:16):
there's a statute, a legal statute that's ambiguous, right?
So if you think about, let me take a step back.
You know, it's July 5th.
We've just had Independence Day.
We're doing a bit of a civics lesson here.
It's not going to hurt a bit.
It'll be very short, but I think it'simportant to kind of frame this up.
The U.
S.
government.
Um, is set up with three different branches, right?
You have the executive, that's the president, basically,and the, the agencies, the departments of, of government.
(01:28:39):
Um, you've got the judiciary, which is the judges.
They basically interpret law and kind of fill in the gaps.
And then you have the legislative,which is basically Congress.
They pass the laws, then, you know, bind the executivebranch that determine what the executive can do and,
and then get interpreted, um, by the uh, judiciary.
Okay, those three branches need to be balancedtogether to ensure separation of powers.
(01:29:04):
That is a fundamental virtue of the American system.
No one branch of government isallowed to accumulate too much power.
It's a way of preventing tyrannyand all kinds of bad outcomes.
Good stuff so far.
Now, Congress passes a law.
That law is going to contain acrap ton of ambiguities, right?
There's no possible way that legislators are goingto be able to anticipate every possible contingency,
(01:29:27):
every nook and cranny and possibility space.
And so you need somebody to fill in those gaps.
You need somebody to say, well, you know, inthis particular situation, it's an edge case.
You know, the famous saying is hard cases make bad law.
Like you don't want to, you know, it canbe really, really difficult to figure out
how to apply the law in certain cases.
The question is, who is actually going to fill in that gap?
(01:29:48):
Until recently, when it came to issues regardingsort of technology regulatory oversight, the
government would defer to the regulatory agencies.
So the argument would be, you know, when it comes to mattersof finance, if you're looking at financial instruments,
if you're looking at stocks, bonds, you know, derivatives,the SEC, for example, just has a ton of expertise.
(01:30:09):
So why don't we just let the SEC interpret what's going on?
What these laws actually mean and then,you know, apply them because they're sort
of like the technocratic experts on this.
Um, the, the idea is that judgesdo not have that expertise, right?
Judges are usually like basically just lawyers whoare dressed up in different suits that like pass
judgment and, and, uh, and interpret the law, but notin these kinds of highly technocratic technical cases.
(01:30:33):
Well, what Chevron deference did was exactly that.
It allowed agencies to be deferred to in theimplementation of these rules of regulation.
That's been struck down.
What this means in practice is we are now going torely on the interpretation of the judiciary, of judges.
And judges are not technical experts, which meansyou want to get rid, basically you're forced to get
(01:30:56):
rid of as much ambiguity in the law as possible.
Congress needs to really be careful.
If they're going to pass AI legislation, theygot to make sure that legislation anticipates
all the possible ambiguities that could arise.
That, frankly.
is very difficult to do.
That is a giant additional hurdle for this whole process.
I don't think it's one that can't be overcome.
(01:31:16):
We've done a lot of work on specifically this question, butI think Chevron deference is a giant, giant challenge or
it's the fact that it's been kind of repealed or, um, sortof like struck down in a sense, uh, is a big, big challenge.
The reality is that, you know, AI is such atech, it's such a technically nuanced topic.
The way scaling works, the way evals work,the way liability ought to work, all these
(01:31:38):
things play into interpreting the law.
And, you know, you can't expect judges to havethat kind of understanding, uh, very quickly.
So this is a, you know, it's a big issue.
Also, you can imagine if Congress is forcedto resolve issues down to insanely high levels
of detail in order to basically like, As Axiosand Axios reporters said, predict the future.
That's basically what they're being askedto do is like predict every possible way
(01:32:00):
that these laws could be interpreted.
Um, if you're going to ask congressmen to dothat, well, you're going to have an awful lot
more of these petty squabbles that are going toarise because now you're having to have all these
arguments at a higher level of resolution and detail.
And frankly, the system was notset up with that expectation.
In fact, historically, It was kind of understood thatwhen Congress writes ambiguous laws, it's implicitly
(01:32:23):
delegating authority to agencies to fill in those gaps.
Now what we're basically saying is going forward, so this isnot a retrodictive thing, it doesn't apply to past laws, but
going forward, yeah, Congress is going to have to do this.
They're going to have to get down to the levelof resolution in detail that regulatory agencies
used to, used to have to result down to.
Um, so this is a, I think a pretty big blow to the U.
(01:32:45):
S.
's nimbleness, right?
It needs the dexterity to respond to things fasterthan the legislative cycle can allow, especially in AI.
So this is a really, really baddevelopment in my estimation.
I think in practice, like I don't know about theinterpretations of the constitution that went into this.
I am not a lawyer, certainly not a judge.
Um, But in practice, the effectsof this are very, very harmful.
(01:33:07):
Um, it does come the same week as well as thecorner post ruling came out, which removed
basically a, a six year limit on the rights ofaffected parties to challenge past regulations.
So it's, it's part of a kind of wave of deregulation,which normally I'm all for, but in the context
of AI, it, you know, stuff is moving so fast.
You need to be able to defer to a fast moving kindof executive cycle time rather than relying on,
(01:33:30):
you know, judges and, and that sort of process.
There you go, that's your civics lesson, and uh,certainly I've learned a little bit, so thank
you Jeremy for being the expert on this one.
Next up, NVIDIA to make 12 billion fromAI chips in China this year, despite U.
S.
So, uh, these 12 billion will come from thosespecialized chips that are, uh, the H20 chips.
(01:33:59):
It sounds like there'll be deliveries of a ton of them.
It seems that NVIDIA will deliver more than 1 million ofthese chips, and they cost between 12, 000 and 13, 000.
dollars.
And that's the suggestion of 12 billion dollars in sales.
So yeah, there you go.
(01:34:20):
Uh, despite the controls, it's still the case andNVIDIA is doing a lot of business in China, but these
chips are about one eighth as powerful as the, uh, verymuch leading chips that NVIDIA is selling in the U.
S.
So, uh, The export controls are having some ofthe intended effect, even though they're not fully
(01:34:42):
shutting down Nvidia's ability to do business in China.
Yeah, I think this is a really interesting kind of, it's astory about the lagging tail of Nvidia's business in China.
Interestingly, by the way, Nvidia is actually doing morebusiness in China than they were this time last year.
Um, but it's, it's as a fraction of their, I mean, they,you know, their income's grown so much, their sales
have grown so much that, uh, as a percentage of theirbusiness, you know, it's, it's dropped by quite a bit.
(01:35:07):
Um, you know, a couple of interesting take homes from this.
First off, you know, the H20 GPU, you can thinkof it as like NVIDIA takes an H100 GPU, so
like a regular top of the line GPU, and theybasically fry some of the circuits and ship it.
That's basically how these things are made.
Um, the same is true of a lot of the other kind ofderivative GPUs that are meant for the China market.
Uh, one of the things that, um, so there's a, aninteresting quote from, from, uh, from Uh, Dylan
(01:35:30):
Patel, who is, uh, one of the guys who runs semianalysis really, really, really recommend this
newsletter if you're interested in AI hardware.
Uh, so he said, uh, that the, although theH20's capabilities on paper are actually
worse than their sort of domestic competitor.
This is the Huawei 910B chip, talked about that quite a bit.
That's the chip that's based on theseven nanometer process that SMIC has.
(01:35:53):
Um, so on paper, the H20 should be a little bit belowthat in practice, it's actually a decent bit ahead.
Bye bye.
Especially it's, it is got better memory performance.
So the way in which the, the kind of area onthat chip is allocated is just a little bit
more optimal than the, uh, the nine, 10 B.
The design of the chip is just better,uh, which, yeah, maybe not too surprising.
NVIDIA's damn good at this and, and Huawei.
(01:36:15):
While they're getting up to up to speed intheir game, they're not quite there yet.
Um, so the expectation here is that thisis going to start to, to change over time.
NVIDIA's H20 GPU is going to fail to continueto compete with Huawei's latest and best.
And, uh, and look, those exportcontrol constraints that the U.
S.
government's put in, the Department of Commerce, uh,I don't expect those to, to get like more permissive.
(01:36:39):
So I think over time, probably what's going to happenis Huawei is going to gobble up more and more of the
domestic Chinese market as they and SMIC and othercompanies like Shenzhen, um, kind of pick up their game
and allow fully domesticated, uh, GPU markets to pop up.
Not too, not too surprising.
It's kind of an obvious, um, uh,result of the export controls.
(01:37:00):
Um, and it was expected in like, as far as I can tellinternally, USG, a lot of the people I talked to,
you know, this is where it was going to go, but itis having a desired effect, it is slowing China down.
And, um, I think, you know, you gotta, gotta takethe W unless you're, you're in video, I guess.
Yeah.
And, uh, just so it's clear, we are not takingsides here, even though we are based in America.
(01:37:23):
Uh, we just are, want to cover this export controlslaw and maybe, you know, we might have personal
feelings on slowing China down and so on, butwe don't want to sound like we are taking sides.
Oh, maybe,
maybe I should be open about my biases.
I, I absolutely am pro slowly China down, but, uh, Yes.
Okay.
Yeah, it's true.
(01:37:44):
We should acknowledge that we have biases and thatmight impact how we cover these things, but also
objectively, these are the facts in this case.
And, uh, another story on those topics, the next oneis that Uncle Sam, aka VUS, apparently relies on manual
processes to oversee restrictions on the Chinese government.
(01:38:05):
So this is about the Bureau of Industry and Security, whichis responsible for implementing export licensing controls.
And it's, uh, going into how that agency is struggling.
Because they have these manual processes going back to 2006and the scale of what they're doing has increased a lot.
(01:38:31):
So apparently the numbers of Chinese entities ontheir list, uh, which include organizations that us
businesses are not allowed to trade with rose from.
2018, uh oh sorry.
218 in 2018 to 787 in 223.
(01:38:51):
So that's 550 additionalorganizations over span of five years.
And so, yeah, I think, uh, it's worth kinda being awareof just 'cause this is another indicator of how big a
deal all of this is and how the US has really committed.
(01:39:12):
to these things.
Uh, and there's, yeah, quite a fewinteresting details on what this agency does.
Yeah, BIS is really, I mean, it's at the Departmentof Commerce, one of the most important branches of it.
It's focused on this whole export control piece.
And um, there was an entry, yeah, to your point, Ithought it was a really interesting stat at the bottom.
(01:39:34):
Um, they look at the, uh, 4, 000 entity list.
applications.
So the entity list is basically the set ofcompanies that are big no nos that you, you can't
sell to, um, just without getting a license.
And so they have 4, 000, uh, applications thatwere reviewed by the BIS over the last six years.
And they say around two thirds were approved.
Um, but, um, They were for 335 billionworth the rest, which are actually for more.
(01:39:58):
So it's a few of the applications, butthey're worth more of the money, uh, 500 and
550 billion or so were denied or revoked.
One of the important things to note too is,uh, uh, my understanding is that for a lot of
these things, there's a presumption of denial.
So essentially by default, you should expectthat you're going to be denied the license
to export, which starts to matter a lot.
(01:40:20):
When you have, um, organizations like the BIS that arekind of undermanned, and so essentially you got a longer
delay between, you know, approval and, and all that.
So, you know, that, that kind of means that bydefault, you know, you're not sending these things out,
which is of course the national security objective.
Um, so yeah, uh, I think kind of agood update on what the BIS is up to.
(01:40:41):
It's kind of like NIST, right?
We're relying on them more and theyhaven't gotten more funding or as much.
Additional funding is they mightneed to deliver on their mandate.
So, you know, be great to see things like Congress,uh, deliver on a little funding package for,
for NIST and maybe for BIS while they're at it.
And speaking of government organizations andprograms, the next one is Telling us what the US
(01:41:05):
government has launched an initiative to addressworkforce shortages for the semiconductor industry.
So this is addressing shortages of workersprojected for the industry by 2030, uh,
that's going to use part of a 5 billion.
(01:41:26):
allocated to the new National SemiconductorTechnology Center that will distribute grants
to different workforce development projects.
And of course, this is very significant.
The U.
S.
has invested a lot in the semiconductor industry of the U.
S.
with the CHIPS Act.
And it seems that, uh, kind of the rate ofuniversity completions, uh, are just not keeping up.
(01:41:53):
Apparently, uh, of the new jobs projectedfor, uh, the industry as it grows, uh, 60
percent are expected to remain unfilled.
So big gap in these specialized skill sets.
And, uh, there's some good stats in the story sayingabout 50 community colleges have introduced or expanded.
(01:42:14):
Programs related to semiconductor technologyand companies like Intel, Samsung, and others
have allocated money for workforce development.
So yet another aspect of the semiconductor industry,uh, that makes it a challenging space to, uh, get into
and why governments need to spend hundreds of billionsof dollars to even give their national players a shot.
(01:42:42):
And the last story, Bridgewater Starts 2 Billion Fundthat uses machine learning for decision making and will
include models from OpenAI, Anthropic, and Perplexity.
Bridgewater Associates in this case will launchthis fund, presumably an investment fund, that
will use AI to make some of the trading decisions.
(01:43:06):
Uh, this is part of a general transitionto using more machine learning.
And to me, it's interesting, uh, in, in partbecause the whole theory of OpenAI Basically since
forever was that once we have a super advancedAI, it will just make all the money, right?
That's the idea.
(01:43:27):
Open AI has run with for a very longtime is they won't be profitable.
And then they'll just make all the money with AI.
And, uh, I guess this could be oneexample of how that might work.
AI will just make all the money viabeing the best investor in the universe.
And we'll see how that actually.
Turns out,
I like how that sentence endedand we'll see how that turns out.
(01:43:51):
Uh, yeah, news at five.
Um, yeah, no, I think this is a really interestingpublic development, uh, in this direction.
So this is, uh, Man, there, there's so much here.
I mean, so there are, um, this is going to start,this fund is going to start with 2 billion of capital.
They've been experimenting with a smallerfund, about a hundred million dollars.
(01:44:12):
So that's chump change in the space, uh, for a littlebit while now, just to confirm this is actually working.
Cause obviously you need to make sure that,you know, you're, you're training in within
distribution and testing within distribution.
So they've actually been using this at small scales.
Um, One interesting note that, so the guy who'srunning this, uh, this is a, um, apparently a
broader venture that's been spearheaded by theirco investment, sorry, co chief investment officer.
(01:44:36):
This is a guy called Greg Jensen,who I had never heard of before.
Um, but he worked apparently at Bridgewatersince 1996 and he actually put his own
money down towards opening eyes seed round.
OpenAI's seed round, mind you, I don't know,that's like 2015, I think, um, like a long, long
time ago, and then was also one of the first,uh, was part of the first money in for Anthropic.
(01:44:57):
So this is a guy who's really, reallytracking the space very closely.
Um, it should be noted, this is almost certainly notthe first firm to experiment with this sort of thing.
You know, when you think about, you know, who's leadingthe way on this sort of like, not just AI enabled, but
basically automated, uh, trading strategies, you know,you think about, Renaissance Technologies Medallion Fund.
(01:45:19):
That's probably the first one.
This is a fund that was set up byformer intelligence community people.
Um, that's notoriously secretive,super, super, super AI first.
They've been doing, you know, things thatgesture in this direction for a long time.
You think about Jane Street as well, you know, a bunchof companies, a bunch of funds in this space that
have been kind of, you know, You would expect wouldbe just as forward leaning if not more so on this.
(01:45:42):
Um, so this is what we know about.
Keep in mind, this is just thepublic stuff that we know about.
Um, there has been interest, there has been interestfrom, I think it was a congressional report published
recently, uh, that was talking about concerns overbasically funds using Uh, AI to, uh, potentially disrupt
markets, uh, increasing the fragility of markets, butalso you can think about in the more extreme limit
(01:46:06):
as these things get more and more capable, right?
What's more useful than an AI model that just, you know,makes predictions about how stocks are going to move.
Presumably one that can actually proactively reach out.
To analysts to get their opinions to inform itspredictions and what, like what's more useful than that?
Well, you know, maybe a model that can do that, uh,but not just to get analysts opinions, but to start to
(01:46:27):
shape their opinions because that can influence markets.
Yeah.
You can kind of walk down that line ofcausality and ultimately giving your
model more and more of an action space.
You get to the point where youstart to worry about these things.
You know, placing shorts on very unlikely catastrophicevents or things like that, and then making them happen.
Um, this is a threat model that, youknow, sounds, sounds out there today.
(01:46:50):
Uh, well actually less and less so I guess butthat um, at least, you know, we know have been,
has been uh, tracked by a couple of people wholet's say would, would be interested in the space.
So, you know, Increasingly, you know, hedge funds arebecoming a source of potential, uh, sort of AGI ish risk
as you, as you see them sort of start to reach furtherand further into inaction space in the direction of
(01:47:10):
fully automated systems, they have an incentive to worklike crazy to just, you know, pump that bottom line.
And, uh, those incentives aren't great in thecontext of technology that gives you more and
more of a potential destructive footprint.
So interesting, nonetheless.
Um, this guy's got, uh, some deep connectionsapparently to Anthropic and OpenAI.
Hopefully that makes them more awareof some of the safety risks here too.
Um, but we'll, yeah, we'll see what happens with thisvery large or not very large fund, this medium sized fund.
(01:47:35):
Um, cause Bridgewater's total assets are over ahundred billion that they're tracking right now.
Yeah, definitely.
As you say, we know.
So I do all this stuff too about hedge funds.
Uh, I think we are all aware of these things, right?
It's all basically the big short.
And that's our last story.
So thank you so much for listening tothis latest episode of last week in AI.
(01:47:59):
As we always say, you can find thosearticles and subscribe at lastweekin.
ai.
You can also reach out to us with your feedback.
We include the emails you can, uh,use in the episode description.
As always, we do appreciate if you share the podcastwith your friends or enemies, and if you review
(01:48:20):
it to give us some feedback, even if it's negativefeedback at the hopefully constructive feedback.
And now we are done and do enjoy this fun.
AI outro song.
Make
it fun, you see, from AI agentsto Chevron deference with glee.
(01:48:46):
Talking Moshi, it's the buzz indeed.
AI agents making moves with speed.
Chevron deference, let's get it clear.
For all you AI fans, the best is here.
(01:49:07):
Last week in AI, here we go Talkin on Java, now don't youknow Gen 3, Alpha 2, so fun Get ready for the wild ride
From the labs to the street, tech news flows Stay connected,see what tomorrow holds We've got your AI updates on the
rise Innovation that'll mesmerize So grab your headphonesAnd join the stream For the latest information Machine
(01:49:40):
learning dreams.
Learning dreams.
Popular Podcasts
United States of Kennedy
United States of Kennedy is a podcast about our cultural fascination with the Kennedy dynasty. Every week, hosts Lyra Smith and George Civeris go into one aspect of the Kennedy story.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com