May 29, 2025 • 50 mins
ChatGPT is amazing at many things…but it can’t seem to figure out poker. Nate and Maria explain why, and talk about the implications for artificial general intelligence. Plus, they discuss Trump vs. Harvard, round two.
Further Reading:
Silver Bulletin: ChatGPT Is Shockingly Bad at Poker
Harvard Derangement Syndrome, by Steven Pinker
For more from Nate and Maria, subscribe to their newsletters:
The Leap from Maria Konnikova
Silver Bulletin from Nate Silver
Get ad-free episodes, and get your questions answered in an exclusive weekly bonus episode, of Risky Business by subscribing to Pushkin+ on Apple Podcasts or Pushkin.fm. Pushkin+ subscribers can access ad-free episodes, full audiobooks, exclusive binges, and bonus content for all Pushkin shows.
Subscribe on Apple: apple.co/pushkin
Subscribe on Pushkin: pushkin.fm/plus
See omnystudio.com/listener for privacy information.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:15):
Pushkin. Welcome back to Risky Business, a show about making
better decisions. I'm Maria Kanakova.
Speaker 2 (00:31):
And I'm Nate Silver.
Speaker 3 (00:32):
Today on the show, we're gonna be talking about how
chat GPT plays poker, i e.
Speaker 2 (00:38):
Poorly.
Speaker 3 (00:38):
If chat gipt is getting ready for the World Series
of Poker like us, then it's got a lot of
homework to do.
Speaker 1 (00:44):
Yes. And after we talk about that, we'll talk about
Harvard two point zero aka the fact that foreign students
are now being potentially barred from attending Harvard University. Harvard
has sued the administration. Anyway, we'll be talking about what's
going on with that and what the implications are for
(01:06):
the future of the United States and kind of research development,
brain power and the US competitive edge. So for people
who don't subscribe to Silver Bulletin, which you absolutely should,
Nate and I both had poker posts this last week.
(01:27):
Mine was about cheating, was about chat GPT, but his
post this week was one of the funniest things I've
read in a while about his attempts to get chat
GPT to simulate a hand of cash game poker. And
one of the reasons I mean, I found this amusing
(01:48):
because I mean, first of all, it fused cards together
for an image. From the beginning to the end, it
was pretty spot on in terms of being spot on wrong.
But Nay, You're also someone who is constantly writing about
how good AI is at so many things, and so
it was funny to me to see it fall so
(02:08):
short of something that actually tests intelligence as opposed to
being able to kind of pull together a lot of
things and spit out an answer.
Speaker 3 (02:19):
There are some reasons why I'm just sid in testing
chatchipt on poker, but they kind of fall into two
big buckets. Right, What is it like poker played in
the context of a real hand or in this case,
a real text based simulation I guess of a hand.
It really does require quite a few skills, right. There
(02:40):
is the pure math part of it. What is the equilibrium,
the Nash equilibrium, the gto strategy that you're solving for.
There's also making adjustments for how other players play. There's
the conversation you're having, the physical reads that you're getting
their stuff, like knowing what the rules are right, which
might seem trivial, but we've yeah, chatchpt also.
Speaker 1 (03:03):
Does not know.
Speaker 2 (03:03):
It seems to be no, it is fun to look.
Speaker 3 (03:06):
I mean, I'm sure we've all had hands where we
like misread a board or I you know, I had
a big hand in the main event at the when
I play back in Florida back in April or whatever,
right where like a guy had like a twenty five
thousand dollars chip that was like almost the same color
as the felt, which is not good chipkit maintenance, by
the way, Seminal hard Rock. I like the WPT, but
(03:26):
that chip kit it's got to be improved, right, But yeah,
you're tired, you make mistakes like that. But and you
also have to have like a lot of short term memory,
which sounds trivial, but you you know, you want to
remember how you arrived at this current spot in the hand, right,
you have to keep crack of all these stack sizes,
(03:47):
which again seems trivial, but like it's a good test
of general intelligence of a certain type, especially a live
poker hand, like I'm asking.
Speaker 2 (03:56):
Chat ept to simulate.
Speaker 3 (03:58):
Right. The other reason is that like I don't think
engineers in Silicon Valley are trying to optimize it for poker.
And the reason that's important is because like, look, there
are benchmarks like math Olympiad problems that these LM's large
language models like CHATCHPT, Claude, et cetera compete on, right,
And they're like, well, we be bragging that we can now,
(04:20):
you know, beat all but the Nobel Prize mathematicians on
x percentage of math problems and things like that. And
there are a couple of issues with that, right. Like
one is that, like, clearly, if you train a transformer model,
a machine learning model on a particular type of problem,
like it can do fairly well or very well often, right,
(04:42):
Like poker is to a first approximation, and it's an
important approximation. Right, But if you have poker dedicated tools,
I wouldn't consider a solver and AI that's a technical
distinction I think might not be that important for our listeners.
But like, you know, if you want to use computers
to play very good poker, then they can play very
very good poker. Right. The question is like, can you
(05:04):
take a text based model and have this organic property
where intelligence emerges and converts from the data set towards
superintelligence without training it on poker specifically. And the answer,
at least as of last week when I did this,
is no, it fails miserably.
Speaker 1 (05:22):
Yeah, and I think it's actually important. One of the
things that came to my mind when I was reading
your piece was that poker has been a benchmark for
AI development well before LMS, Right, It has been kind
of the gold standard that teams all over the world
have been working on for decades because it is a
(05:46):
much more complex game in many senses than chess, then
even go, and it's a game where, you know, if
computers can actually manage to outthink human players on a
broader scale, that would mean something in terms of kind
of broader intelligence capabilities. And before LMS were developed, that
(06:08):
had not happened. Right, There had been computer programs, kind
of AI algorithms that have been able to beat heads
up one on one opponents in poker. But when it
came to full ring games, So for people who don't
know poker, that means basically, when you put it in
a rich environment with you know, six players, eight players
and you have a computer there, it was still not
(06:31):
quite there. And one of the you know, one of
the reasons why poker so interesting is it's not just math, right,
especially when you have multiple players. There's so many dynamics
and there's actually a very thorny problems for AI, which
is that if you program something to follow an algorithm
right to be GTO game theory optimal, then it can
(06:55):
be exploited by a human if the human figures out
the GTO strategy. Now, if it were another human, then
you'd adjust immediately right the moment the human adjusts, the
other human would adjust and kind of figure it out.
But if you're an a if you're a computer, and
you build in that adjustment parameter, what ends up happening
is that the model adjusts too much and too quickly,
(07:18):
and so that lacks kind of some of the nuance
and flexibility that marks that's kind of the hallmark of
the great poker players, right, kind of that ability to
into it when it's time to slightly deviate. And I mean,
it would be terrifying but amazing if a model could spontaneously,
(07:39):
like an LLM, could actually spontaneously figure that out and
so far, I mean, instead instead of doing that, Nate,
you know, your model decided that the person who won
the hand actually lost the hand, right, It just got
basic things completely wrong, and you just you know, obviously
we've talked about pe doom a lot on the show.
(08:00):
I think you and I are both worried about that.
But when I see this poker stuff and like, oh man.
Speaker 3 (08:05):
Yeah, I don't know how much to update. But look
my say in the post, I mean, like my appreciation
and expectations for large language models have increased a lot
over the past year. I had a pretty good experience
working with them when I was building the NCAA tournament
model that we host silver bulletin and just helpful kind
(08:27):
of Swiss army knives or like annoying data related tasks.
It's late at night, you forget the Stata command for something,
You're like, right, give me the right command and write
five lines of code, and like usually it works, and
it kind of now shows you like the chain of
reasoning and like what it's thinking. Right, And but you know,
people are saying that we're going to have like AI
(08:50):
revolutionizing all programming or you know, well, first of all,
the big claim is going to have AGI, meaning that
a machine. It's a little ambiguous when people say, does
this mean like chat GPT itself is AGI? Right, Well,
clearly not. It's very far from human being in some things, right,
(09:11):
Does that mean that a combination of transformer or machine
learning based technologies is or zooming out the definition of AI.
It's really AI still, right, so that you have chechiptiza
seventy percent of things well, and then you specialized applications
for twenty percent, and then there are ten percent where
you're not doing it very well. Like that would still
(09:31):
be a big achievement, very disruptive achievement. I'm not sure
it would kind of quite count as AGI. Like one
thing you could do is, like some of these new
models will say, okay, I can detect that you ask
for something that requires mathematical precisions. So let's take a
simpler example. Right, Let's say I want to take the
(09:53):
distance between pick two cities, Boise, Idaho and Tampa, Florida. Right,
that's pretty easy to calculate mathematically if you look up
the latlong coordinates for Boise and Tampa, and then there
are various formulas that.
Speaker 2 (10:09):
You can use.
Speaker 3 (10:10):
Right, So a pure data set where it's just crunching
texts might not find that answer because there's not necessarily
a lot of examples of like what's the distance between
Boise and Tampa? Right, Whereas you would have that for
like La and new York or something. However, you can
set up kind of agentic meaning agent ais where they're like, okay,
(10:31):
well this person asked a question that triggered a routine
of mine. Where now I'm going to go and I
know to search the internet for the coordinates or maybe
it's stored somewhere right of Boise and Tampa. And then
the formula. I know the formula and I can apply it, right,
so like you have the scaffolding. It's sometimes called a
different processes on top of one another. And I guess
that kind of I guess that counts as I mean,
it's kind of how humans solve problems, right, They're like, oh,
(10:54):
and now I have to go look something up, right,
It's still I think subtly different. So one implication I
think this has is artificial general intelligence versus super intelligence ASI,
sometimes called where there's like an explosion of intelligence just
from kind of mining text or other data sources. I
(11:17):
have actually become a little bit more skeptical of that,
or I think it's presumptive to assume it can find that.
I mean, like, let me give you another example, right,
like one I had an idea for either a silver
bulletin post or maybe even a risky business segment, right
where like I asked, chatchipt deep research, design a meal
(11:39):
with foods that have no clear precedent in other countries, right, huh?
And I thought, and the idea was that, like, you know,
at some point somebody invented pizza or spaghetti or whatever else, right,
or or I.
Speaker 1 (11:54):
Mean at some point somebody invented bread and figured out
that you can take flour and like bake shit with it.
It's amazing.
Speaker 3 (12:01):
Yes, you have to be able to buy these ingredientstead
of Trader Joe's or a Whole Foods. Right. I was
just going to like hire professional chef and have a
dinner part and ask how these creations went, right, And
it's like and all they recipes were like here's salmon
with miso paste, right, really creative and things like that, right,
and like it didn't have any ability to like extrapolate
(12:21):
beyond the data set. I'm sure these preparations are fine.
It was like their little chefy, meaning like things that
like have one too many ingredients, right, and like and
I don't, yeah, look so bad chefe.
Speaker 2 (12:35):
Yeah.
Speaker 3 (12:36):
And meanwhile, I've been like I've been interviewing candidates for
like a sports assistant position at Silver Bulletin, and one
of the questions is, how are you using AI? These
are very bright I want to say kids. A lot
of them are are recent college grads, you know, your
early twenties to mid thirties, basically right, and very technically
(12:56):
proficient people. And asked them how they're using a GI
and their experiences like kind of similar to mine. They're like, Yeah,
for a certain task, it's very helpful. For certain task,
it's somewhat helpful for certain tasks, not helpful at all.
I can get you in trouble, right, which again contrast
with the experience of people at the AI l apps
to say, yeah, it's going to like displace all programming
(13:20):
within a year or two.
Speaker 2 (13:21):
I mean, I know you should weigh in here.
Speaker 1 (13:23):
So I have an interesting kind of anecdote to add
to this that's not poker related, but that talks about
kind of some of this nuance and complexity and the
fact that you know that all of these lms still
are wanting in certain major respects. So I had the
chance last week to speak to the CEO of one
(13:46):
of the major AI companies. I won't name which company
it was, but one of the big ones. And someone
had asked him earlier, what was something that was surprising
about the way that this AI functioned, and he said,
you know, one of the mistakes that people is kind
(14:10):
of they trust it, And what we know is that
it's remarkably accurate ninety percent of the time, right, and
then ten percent of the time it isn't. But the
ten percent is getting harder and harder for humans to
spot because it doesn't make errors in a way that's
intuitive for people, right. It makes errors in different ways.
(14:30):
It's like computer errors AI errs, that's not the way
that the human mind makes errors. He's like, but like,
it's fine because we have really smart people who can
figure out where it's making errors and can work with
it and can kind of calibrate it and help. And
so my question, my follow up question, was, Okay, you
know what happens when the new generation that's being brought
(14:52):
up with this AI comes up and they've been educated
with it and they've been using it the whole time,
and they didn't necessarily get the same education that we got, right,
because they aren't the incentives are different, and they're not
getting the deep knowledge. They're not actually able to figure
out what are the programming errors because this person was
talking that, you know, some of the biggest potential of
(15:14):
AIS and like programming and biology and kind of those
those types of things, and I said, well, what if
that person doesn't have the neuroscience and the biology background
or the programming background. And what the CEO said was,
this is a major problem, and what we're hoping is
that we can develop even more advanced AI to help
and to help fix the problems that AI is causing.
(15:36):
Because he was like, yeah, this is a big problem
and we're not quite sure, like we're hoping for the best,
but we don't know. And the fact that this was
our conversation didn't exactly inspire me to heights of thinking
this is going to replace intelligence or become AGI or
what did you call, Nate the other ASI artificial super intelligent? Yeah,
(16:00):
artificial superintelligence. So it didn't inspire me that that was
going to happen, and instead it was like an uh
oh moment where like, what happened when the humans who
are kind of helping push it along and make it better?
When that generation, like when they age out and they're
new people out there who who lack that sort of
(16:22):
skill and expertise and who grew up with AI that
is ninety percent accurate or even ninety five percent accurate,
but they can't. It's harder and harder to spot that
five and ten percent. And that to me was actually, like,
it's a worrisome thing, and it's clearly worrisome to the
people who are developing these technologies as well. And so
(16:44):
that I think is something that dovetails with what you
were talking about here. What happens Nate if all of
a sudden, all poker players are training with chatchipt right,
and that's that's actually great for us, but not great
for the future of poker.
Speaker 3 (17:02):
And we'll be right back after this break. I use
large language models for tasks that I have strong domain
knowledge over and could do myself. But they're a labor
(17:23):
saving device. Often they say substantial labor, right, Like I
use them to copy edit articles I posted the newsletter,
and I you know, I'm a fairly proficient user of
the English language and like for programming, right Like you know,
if you're like using AI to like design a website
and some of the functionality breaks, right, that might be okay.
Speaker 2 (17:45):
You can patch it and fix it. Right, if I'm
doing things.
Speaker 3 (17:47):
Involving modeling, I'm trying to project the value of basketball
teams or football teams, or basketball players or football players. Right,
and there's a bug where all of a sudden, like
one of the worst players in the league is rated
as being very good. Right. I mean, and you see
this in poker, where like you know, AI is relying.
You get punished in poker for sloppy application of imprecise
(18:11):
application of heuristics. Like one of the things like the
AI did so at first, I had it simulate just
one hand, and it had like a backstore for each player.
It fucked that up, right, then like Oka, I'm gonna
use deep research now and have it simulate like an
orbit of eight hands.
Speaker 2 (18:26):
So that was a little better.
Speaker 3 (18:27):
It took more compute time, right, Some of the hands
were decent, but it you know, it didn't know how
to like read aboard correctly and keep track of stacks.
But like, but when it was over able to overcome
those errors, it also had poor strategy and it gave
various excuses. One thing it said is like, well, the
quality of text based content on poker on the internet
(18:49):
really sucks, Right, it's aware of that is it says
it's worse than go or chess, although when I talk
to people who know how AI's played chess, it's also
a disaster. Even something like wordle I heard from a user.
You know you think wordle is a word game and
it does very well, but like it can't quite figure
out the strike sure of the problem, right, it kind
(19:10):
of was learning a little bit more by by a
rote And to be clear, like this is a very
good strategy for like many types of problems. And also
if you were to say, okay, here's a bunch of
we bought data from PIO solver or gto wizard. These
are solvers if you don't know listeners or a database
of high stakes hands. Like if you trained it on
that and then had GPTs say okay, poker question, I'm
(19:33):
going to call them especially trained database, then it would
do well. And maybe if people are criticizing its for
on poker, then the labs will start to do that. Right,
but it's still kind of like slightly cheating from the
standpoint of super intelligence. Also, it says like when you
asked me to do a whole bunch of things at once, right,
So I told it, give me characters, give me eight
characters who have like they were all kind of like
(19:55):
ethnic stereotypes, right, it was.
Speaker 1 (19:57):
They were very funny.
Speaker 3 (20:01):
It was kind of yeah and great. It was kind
of at the one hand, so half the players were women.
That was very feminist of it, right, But then they're
all ethnic stereotypes of different kinds, right, and so it
was kind of both woken and unwoke, right the opposite. Yeah,
it was kind of good about like matching players playing
styles with with their actions, but like, but it couldn't
(20:24):
keep track of stack size. It's like, yeah, keep track
of stacks. I have to store a bunch of stuff
in memory, and then when you ask me to do
a whole bunch of things at once, right, and like
you know, and even including like so in the hand
I show a sylver bulletin it like it misreads the board,
doesn't recognize that a pair of nines has a higher
two pair. If you ask it that out right, then
(20:46):
it thinks about it more and gets the question right, right,
But like it loses track of things in the context
of these scaffolding situations where it has to keep track
of a bunch of things at once and it doesn't
know it doesn't know what to prioritize right. Like you know,
you can make a lot of mistakes in poker if
you don't know which hand be twitch hand. That's a
more elementary mistake than anything else.
Speaker 1 (21:08):
Absolutely, So there are a few things that stand out
to me here.
Speaker 3 (21:11):
One.
Speaker 1 (21:12):
I mean, it's a computer, right, like we humans have
working memory capacity problems. It should be able to keep
all these things like this is what it's good at.
But the problem that you're kind of that you're illustrating
is it doesn't know how to prioritize right, and that
that is actually that is a major problem. And also
(21:32):
when we keep saying thinking. But one of the first things,
so when I was just learning poker, one of the
first lessons I had with Phil Galfon, who is one
of the people who kind of I worked with and
who taught me a lot. I remember very early on
he said, you know, I can give you a bunch
of charts and a bunch of outputs, and you can
memorize them and you'll be a very decent player very quickly.
(21:55):
Like I know, I can give you this shit, you
can memorize it, you can spit it back at me.
You'll be fine, He's like, but I don't want you
to do that, because that's going to make you a
fine poker player, and you'll do fine in the short run,
but you'll never be a great poker player because you
have no idea why you're doing it. You're not actually thinking,
you don't understand. You've just done a rote memorization, which
(22:16):
is might make you more money in the short term,
but in the long term is actually going to be
detrimental to you because you're going to do stuff unthinkingly
because you've memorized it.
Speaker 3 (22:25):
He said.
Speaker 1 (22:25):
What I want you to do is instead think through
things and figure out, Okay, why right? For every single play,
why am I doing this? Why would I play this
hand and not this hand? Why would I raise this
type of hand and not this type of hand? Why?
Speaker 3 (22:41):
Why? Why? Why? Why?
Speaker 1 (22:43):
And that's something that to this day has stayed with
me because it's such an important thing to remember when
you're making a decision. Right, And this doesn't even have
to be poker, It can be any sort of decision.
Why am I choosing this action? Why is it better
than all of the other actions? Right? And if you
even if you feed you know, piosalvereign gto Wizard outputs
(23:05):
two lllms, they'll be doing some rote of memorization. You know,
at this point they're not understanding the why, and so
that will lead them still to make big mistakes, which
happens when you just learn solver outputs, no matter how
sophisticated those outputs might be and how correct they might
(23:26):
be in one specific hand, in one specific spot, if
you don't understand the why, you're either going to overgeneralize
it right or misapply it like you're going to screw
it up because you don't understand the underlying reasoning. And
there's the difference between looking like you're thinking and actually thinking.
(23:46):
And yeah, you know, as a human you have other pitfalls,
and even if you're you know, thinking through things, you
can mess up, So you mess up in different ways.
But I think that's a really important distinction and probably
one of the reasons why AGI is not as close
like poker. Actually illustrates in a very practical way, why
(24:08):
the notion that you can just have a lot of
data and all of a sudden have this you know,
flurry of insight might not be as I guess, as
intuitive as one might think.
Speaker 3 (24:19):
When one thing, Chatchip you told me when I asked
to like audit itself, is like, well, poker's very difficult
because it's adversarial. You know, understand that there's some game
theory there, but it's adversarial, and like there's no one
I mean, there is like I guess some superstructure of
like a solution for all poker hands if you get
(24:39):
very zoomed out, right, but like but like subtle things
when you have to calculate like an adversarial equilibrium on
the fly basically, and it's that's very difficult.
Speaker 1 (24:52):
Yeah, I mean think about not even poker, but think
about trying to do a game theory solution for multiple players.
Speaker 3 (24:59):
Right.
Speaker 1 (24:59):
It's hard enough trying to do a payoff matrix for
two players when you're really trying to think through it
and you're trying to think of all of the different
payoffs and figuring out, you know, how to exactly do
you wait them, how exactly do you structure that. Now
when it's three players, four players, I mean, it's incredibly
difficult to do that accurately. And so even if we
(25:19):
just zoom out from poker, like this is just an
incredible a very tough problem, even if you understand game theory.
Plus I mean, I think that anyone who has used solvers,
and anyone who's talked about this understands anyone who doesn't
even play poker but has worked with algorithms, the common
saying garbage in, garbage out is absolutely accurate.
Speaker 3 (25:39):
Right.
Speaker 1 (25:40):
The solver works based on what ranges of hands you
as a human put in there, right, and what responses
you allow or don't allow, right, And yes, it will
come up with an equilibrium strategy. But if you were wrong, right,
if there's a player who's playing a totally different range,
if there's a player who's playing a totally different strategy,
(26:02):
all of a sudden, your solution means nothing. And as
a human you can figure that out, and you can
kind of make adjustments from baseline, right, if you understand
what the baseline strategy as you can adjust as an LM,
as an AI who is learning that but doesn't kind
of have that nuanced experience at least at this point.
I don't pretend to know what AI is going to
(26:24):
be capable of in you know, in five years and
ten years, but at least at this point they're not
capable of making those sorts of nuanced extrapolations and figuring out, Okay,
what were my inputs accurate, right, or were my inputs
not quite accurate.
Speaker 3 (26:43):
Look, one thing human beings are good at is that
human beings are relatively good estimators, and chatchipt can be
sometimes if it's like, Okay, take all this text on
the web and kind of give me the average of that,
Like there are some applications where it's pretty good. But like,
you know, the other day, I was getting getting a
drink with my partner and a guy who looked like
(27:05):
he was homeless comes in and like hands like the
hosts slash barked like a note saying call nine to
one one or something like that, right, and you kind
of have to make this judgment call about like is
this like a paranoid schizophrenic or is someone actually in danger?
And like I think was pretty clear. I mean they're
(27:26):
probably you know, two bartenders and eight customers clear to everybody.
Like this guy I think is not like acutely in
danger or anything. But I'd never quite experienced situation like
that before. And like the fact that we can use
like our general intelligence about like human behavior Yeah, Like
I think everyone made the right decision that we're not
going to call the police either to rat on him
(27:47):
or to say he was in danger. And like you know,
and that kind of thing is is is hard for
language yells to do, you know.
Speaker 1 (27:57):
No, I mean I think in general, like this just
illustrates a really good point, which is that humans, even
not the smartest humans, are much smarter and much more
capable in very basic ways that we take for granted
than a lot of kind of super intelligences. Right, Like
it took forever for a robot to be able to
(28:18):
pick up an egg, right, It's something that we don't
even think about, but this was like a robotics problem
that was absolutely unsolvable, Like how do you get a
robot to pick up an egg without without breaking it?
And we do it without just without thinking about it.
And we make judgments all the time, just silly judgments
(28:38):
that we don't think twice about that are so easy
for the human mind, but that are that we don't
even realize we're making right things about safety, things about
just perceptions of the world. So I think that you know,
humans are much smarter than than we think in like
in dumb ways, if that makes sense, Like in waste
(29:00):
that seemed like they're not hard problems, but those are
actually some of the hardest problems to solve.
Speaker 2 (29:05):
I just think of.
Speaker 3 (29:05):
There being four levels, right, Like one more AI performance
better than any human, two where AI performs better than
like all but expert level humans, Three where AI gives
kind of a passable substitute performance but like you know,
not quite professional high grade level, and four where it's
just inept right to the point of being comically inept.
(29:28):
And like, you know, my heuristic is that like within
a few years we might have roughly an even divide
between those four and I'm counting, by the way, tasks
that involve manipulting the physical environment, which I think AI
will mostly be pretty bad at. Right, I don't know
how much I should extrapolate from the poker example. It
was just so incongruent with these predictions of like imminent AGI, right,
(29:54):
where a year ago would have said, okay, yeah, of
course suck at poker. Right, it's not really trained on
this and it's a hard problem, and ha ha ha, Right,
But like if you want to have these complicated, like structured,
nested tasks, like I'm much less worried now in periods
of oh, let's say five years of AI like being
(30:15):
able to build like atiscal model to like forecast elections
or the NFL or whatever else, because that like just
requires like a superstructure of lots of little tasks, all
of which are fuck upable, And if you get the
superstructure wrong too, then you're kind of just drawing dead.
To use a poker term, right if any if you
(30:35):
have to chain together twenty steps and any step, there's
a ninety percent chance you get it right. Well, point
nine to the twentieth power means you're almost.
Speaker 2 (30:42):
Sure to fuck something up.
Speaker 1 (30:45):
Yeah, fuck upable is a really great word to describe this.
Nate to kind of to sum this up, you did
ask chat GBT why it was bad at poker? What
did it tell you?
Speaker 3 (30:55):
Yeah? No, I mean it's said various things. It said, Look,
you stressed me out by having me having me have
to make up these players and things like that.
Speaker 1 (31:03):
Trust it out, You trusted.
Speaker 3 (31:06):
It said the data that it trained on on the
internet is which seems realistic.
Speaker 1 (31:11):
Yeah, it's a data problem. It's a yeah, it's not me.
Speaker 2 (31:14):
It's you. It's it's complicated.
Speaker 3 (31:18):
Inform about like how like it doesn't realize how important
stack sizes are because it's just a bunch of tokens.
Like all the input you give it is generated into tokens, right,
and like the word the is not very important. The
fact that Maria has fifty two thousand dollars in chips
is very important, right, And it doesn't know how to
distinguish those from its transformer architecture. So like, look, it
(31:41):
is good at like it is good at verbal reasoning.
Like again, I I, you know we are talking about
we don't other segments where we're like amazed by AI progress.
Like it's very good at verbal reasoning, or at least
faking that. But in ways I if it's faking, it's
doing a pretty good job. Right, Like it's very good
at verbal stuff for the most part. Right, And AI
(32:02):
is traded on poker, are very good at mimicking Selver solutions,
right and like and so like and machine learning can
get you a lot of ways when you have good data,
but like when you don't have the data in the
training set and it's not seeming to extrapolate very well,
and then these complex tasks that require more and more
compute like, I have not been impressed by deep research,
(32:23):
which is where it goes away and says I'm going
to perform several tasks for you that require deeper thought, right,
and then it takes fifteen minutes you come back and like,
are you fuck this up? Right? And like, anyway, so
I think it has affected my priors a little bit.
Speaker 1 (32:36):
Yeah, that's that's really interesting. And I love, by the way, Nate,
that you know, even though that this is an AI
which is supposed to kind of be better than humans
in so many ways, that use such human excuses for
why it fucked up. You know, you stressed me out,
you gave me that data. It's not my fault.
Speaker 2 (32:53):
This is hard.
Speaker 1 (32:55):
And on that note, let's let's take a break and
talk a little bit about Harvard and the other type
of intelligence, human intelligence. Nay, We've talked on the show
(33:16):
a few times now about kind of the crusade that
Donald Trump seems to have against higher education, specifically the
Ivy League and even more specifically Harvard University. Last time
we spoke was when basically the administration had threatened to
pull funding if Harvard didn't comply with a certain set
(33:37):
of demands. Harvard said fuck you and sued and yeah,
and tried to go that route, and the funding was frozen.
And then the administration last week, so we're taping this
on Wednesday, the twenty eighth of May. So last week,
(33:58):
the administration decided that it wanted to punish Harvard even
more because you know, they didn't like the fact that
Harvard wasn't complying, that it was being defiant, that it
was taking them to court, and so said, hey, Harvard,
you can no longer have foreign students. Anyone who has
visa's effective immediately. Those are going to be revoked. So
(34:19):
Harvard's class of twenty twenty five that's graduating next week
is going to be your last basically foreign student class,
and we're not going to be granting visas anymore to
those students. Harvard has sued again, so now we have
a second set of lawsuits. The initial court cases have
frozen that. But then after that happened in the last
(34:43):
few days, the administration said Marco Rubio actually said that
we are going to freeze interviews for all foreign student visas,
not just Harvard. So they not only you know, went
after Harvard, but doubled down and said, you know what,
you can sue us and you can freeze this. But
if we don't grant the interviews, last laugh is with us.
Speaker 3 (35:06):
Yeah.
Speaker 2 (35:07):
Were they likely to win the lawsuit or are the
likely to?
Speaker 3 (35:11):
Yeah?
Speaker 1 (35:11):
I think so. I think that Harvard is likely to
win the lawsuit. Yes, However, from the lee I'm not
a lawyer, from the legal analysis that I have seen,
they're likely to win the lawsuit. But Nate, let's remember
that that requires first of all, a lot of times, right,
because how many times can we appeal this? And ultimately, well,
if it goes to the Supreme Court, it ends up
(35:34):
being these days less of a legal issue and more
of a political issue.
Speaker 3 (35:39):
Look, I don't think the Supreme Court is likely to
be very sympathetic to Trump on this type of issue,
in part because, like you know, clearly they are somewhat
transparent about like they are not doing it for like
some security concerning and they're doing it because they don't
like some of the speech that Harvard is making. But
it's almost certainly First Amendment protect.
Speaker 1 (35:59):
Yeah, exactly, and I actually think that the So I
don't know if you read Stephen Pinker's op ed in
the New York Times, I don't remember what the op
ed was called, but he termed this Harvard derangement syndrome, right,
kind of playing off of Trump arrangement syndrome and full disclosure.
Steve Pinker was my undergrad advisor, so someone I know, well,
(36:19):
someone i'm you know, I'm still close with, so I'm
very sympathetic. But I thought it was an excellent way
of framing what's going on. And he is someone who
has attacked Harvard. He's a tenured professor at Harvard, but
he's attacked Harvard many, many times for not standing up
for free speech, for not protecting conservative viewpoints, for being
(36:41):
a little bit too woke. Right. He has been one
of the main people who said, hey, like, we have
an issue here. But this his op ed was, this
ain't the way, right, this is not the way to
solve this problem. And you're actually attacking the students who
are some of my most critical, thinking, least woke people
who are on campus.
Speaker 3 (37:01):
But yeah, look, I mean I kind of want to
zoom out and say, like, what is the equilibrium here,
Like what is a Trump administration trying to achieve? What
is Harvard trying to achieve? Like I support maybe some
listeners will get pissed off, right, you know, I think
clearly some of these colleges are not complying with like
the spirit and maybe the letter of the Supreme Court's
(37:24):
affirmative Action decisions, right, and so like, I wouldn't mind
if like the Trump administration's like aggressively enforcing the law
on those or things like diversity statements, which are enforced
political speech. Although Harvard's a private organization, so it's a
little bit more complicated, right, But like I you know,
I don't it's such a blunt tool to like target
(37:46):
foreign students. And even if you hate Harvard, you know,
as a patriotic American, I want the best and brightest
from all around the world to come here and contribute
to our economy and pay our taxes and everything else, right,
And like it's just like purely destroying the value of
these very bright students from helping our economy.
Speaker 1 (38:04):
Yes, it's destroying our intellectual capital. So, by the way,
when we're taping this right now, I'm actually in Hongo,
and I just found out today that several universities in
Hong Kong have offered blanket admissions to all students who
can show that they were admitted to Harvard and can
no longer go because they are a foreign student. They said,
you don't even have to apply, you can just come here,
(38:24):
and these are some of the best universities in Hong Kong.
And that's super smart, right, Like, universities all over the
world should be doing stuff like that right now, because
what the administration is doing is kind of bankrupting at
the future of kind of intellectual development in the country
by saying, oh, foreign students, you know, we've we've just
frozen all these applications. So to me, like that's super smart, right,
(38:47):
Like that is the we've we talked about this several
months ago, Nate, Right, what can China do to capitalize
on the moment it's doing it. It's being like, come here,
like you don't even have to apply, you don't have
to do anything to show us the letter that showed
you were accepted and welcome. Come.
Speaker 3 (39:02):
Yeah. I went to London School of Economics for my
junior year and if I were them, i'd be doing
cart wheels. And they always got a lot of foreign students.
Speaker 1 (39:10):
Right absolutely, I mean I think everyone should be doing
this right now, and I just do not see I
mean a lot of these things. At this point, it
just derangement syndrome seems right, because there does not seem
to be any strategic value to this. There's not any
any value in terms of trying to get at the
types of problems that the Trump administration says it wants
(39:30):
to get at, because it's not even addressing that right,
Like at the originally anti Semitism was the pretext for this,
and this is just like so far divorced from it.
Everyone is having their visas revoked or not granted, or
their interview is not even granted. So it's just, you know,
the pretense is now crumbling. We always knew it was
just a pretense, but at this point, it's just not
(39:51):
accomplishing anything other than to further could of create this
absolute chasm in higher education and future brain power, future
research ability, future creativity of the United States basically mortgaging
its future right saying that we were going to give
up one of the biggest, if not the biggest edge
(40:13):
that the United States has had historically, which is kind
of this brain power creativity place where people can can
develop and find support for for for their ideas, and
now that is not possible and if this actually stands, right,
(40:35):
if foreign students are not allowed into US universities, because
right now, like I said, this interview process being paused
for everyone, not just Harvard. I mean, that is going
to be just detrimental in so many respects to what
happens here.
Speaker 3 (40:53):
Yeah, the next So Harvard is not a very sympathetic
case for reasons that are partly they're full right, but
like you know, so far, the Trump admin has been
pretty smart about kind of which schools they they pick on.
I mean, I don't know what their in game is
here or what Harvard's is really you know, JD. Vans
had it tweet. I don't know where he was like,
he's oddly transparent sometimes about his thinking, and he's like,
(41:16):
well look just like, look, these colleges aren't serious about
complying with the law citing like the students for fair admissions.
I think it's called which was the affirmative action decision,
which I do think they've kind of not been serious
about complying with in some cases case by case basis,
and so like, unless we kind of show real muscle,
then what can we do. We have to fight them
because they're in the wrong, and so we're overreaching. It
(41:38):
is a subtext of that, right, which is like maybe
not crazy and they have a real leader because Harvard
isn't that sympathetic perceptions of higher education that's gone way down,
so it's like harder to marshall public opinion on the
side of this, right, and like and like, you know,
I saw some Harvard professors on Twitter. Ever we can
(41:59):
be like this is the greatest threat to American I'm like,
please don't say that, because it's it's it's maybe more
of a threat to American competitoris and not sure it's
a threat to democracy per se.
Speaker 1 (42:09):
Right, No, I think there are bigger threats to democracy.
Speaker 3 (42:13):
But like, but what's Harvard's end game? Like I think, ok,
if they could go back and tone down some of
the things say down for the past ten years, and
I think they maybe would, but like now, I mean,
if they back down, then they look weak and that
also might hurt their recruitment own nothing, they'd have real problems.
But like, what if you're the president of Heart What's
(42:34):
name Alan Garber? Is that right? Yep? If you're Alan Garber, Maria,
what do you do right now? And you know Harvard?
Speaker 1 (42:40):
Yeah, yeah, I mean, you're in a horrible position. I
think you're doing what you can, which is trying to
stand up to the Trump administration. So Alan Garber is
trying to actually walk a very fine line. He's trying
to stand up to Trump, and he's you know, spearheading
these lawsuits, and he is leading committees into the claims
(43:03):
of anti Semitism and kind of and discrimination on campus,
the anti wokeness. So he is actually trying to address
all of these things in different ways. And I have
no idea, you know, how successful that will end up being,
but it is. It's a it's a tough you know,
he's been put in a really tough spot because obviously
(43:24):
he hasn't been president for long. The last president departed
on lessons tellar circumstances, as did a number of presidents
of Ivy League institutions, and you know, you're picking up
the presidency at a very difficult point in time where, yeah,
(43:45):
you have to acknowledge that Harvard has made mistakes, right,
not just Harvard, a lot of universities I think were
trying to be I don't even want to use the
word woke, because I don't think that's accurate. Here but
we're becoming, you know, places where people were afraid of
voicing their minds. And I wrote, I think almost a
(44:07):
decade ago, I wrote a piece for The New Yorker
about basically, you know, the problems in academia that come
from not having enough conservatives right that it actually has
trickle down effects in research and the types of things
that are being researched in the findings that are allowed
to be voiced, and that it can be incredibly problematic.
And so I think you need to acknowledge that, and
(44:30):
those are things that need to be rectified. At the
same time, you are currently fighting this battle against you know,
Donald Trump, who's trying to destroy higher education in the
United States, and so that is a you know, overriding priority,
which doesn't dismiss all of the things that have gone
wrong and all of the things that are wrong with academia,
but it's just a it's a kind of this fine
(44:52):
balance where you're like, yeah, I know I'm not sympathetic
because I've fucked up and I've done all of these things.
And yet right now I'm one of the people best
positioned to fight this, and so let's just try to
figure out how to prioritize this and how to not
spread ourselves tooth in so that we can actually protect
higher education and then start to remedy the problems that
(45:14):
we have internally, because I do think those are problems
that need to be remedied, right, Like, you can't just
be like, well, now we're all united against Trump, so
let's forget we ever did anything wrong. I think that
all of these things you remember, Nate, like less than
you know. Like six months ago, I was railing against
Harvard because I was really pissed at the way they
were reacting to a lot of things. Now I think
(45:37):
now I'm supporting it because they're being attacked. It's the
classic psychology thing, right where you can actually unite a
lot of people who who criticize you when you attack, right,
all of a sudden, these various factions, when they feel
these life threatening attacks from the outside, all of a sudden,
these factions start uniting. And I think that's what's happening
at Harvard right now, as it should be. But we
(46:00):
still have to realize that, you know, there are all
of these different issues and it's a really I mean,
it's a shit show. That's a technical term.
Speaker 2 (46:08):
Nate yeah, Look, I'm not afraid to use a term woke.
Speaker 3 (46:12):
I mean, you know, critical race theory and intersectionality, all
these things kind of come from an extremely academic context, right,
And Look, the thing I'm probably most concerned about is
the fact that I think the quality of academic research
is often has lots of problems, but like political bias
(46:32):
in predictable directions is among those problems. I'm trying to
think of a politically experienced solution that the Trump administration
might like and that people would find tolerable. On the
Harvard side, Right, we'll collect data on the political affiliation
of our students and faculty. We will make concerted efforts
to I'm not going to try to frame this right
(46:53):
conservati efforts like higher conservative scholars. Right, maybe you have
like you know, if you have an ethnic studies department,
you have a conservative stay.
Speaker 1 (47:00):
No, I mean that isn't that you know exactly what
people have been trying to reverse by kind of with
the affirmative action rulings, right, that you have quotos for
certain types of people. So I think that the better
way to do it is basically having blind hiring, just
like you have blind admissions. You are willing. You just
don't you don't ask any of that, and you don't
(47:22):
disqualify people because they have conservative leanings, right, and that
it's not even it's not a thing because you don't
really care, Like I don't care if my uh, you know,
neuroscience professor, Well maybe not neuroscience as insofar as I
study psychology, but if my astrophysics professor is, you know,
(47:43):
what their political views are, as long as they correctly,
you know, as long as they're one of the best
astrophysicists in the world. So so I don't think that
that's you know, that what you're proposing is affirmative action
for politics.
Speaker 3 (47:57):
I'm just saying, like it's slightly stupid, but like maybe
maybe jd.
Speaker 2 (48:00):
Vance would like it.
Speaker 1 (48:03):
But but you're saying, like do something that he would like.
Columbia tried to do everything and had all of their
funding frozen anyway, right, So like this is the problem
that like actually doing trying to appease is not the
correct play here. I think that's actually the completely wrong
game theoretical way to respond here. That's because it's not working.
Speaker 3 (48:24):
And that's a meta thing across everything that the White
House does. You see it playing out Ontari's too, where
on the one hand, they reverse course and undercut themselves
all the time, right, you know, and they're sloppy enough
where they probably maybe if they're doing this right, they
could have had more likelihood of success in the courts
than they have. Right, on either hand, they don't necessarily
(48:46):
hold up their quid pro quote. This is kind of
very transactional and explicit, right, So they're not even like
presenting clear enough information to make themselves easy to bargain with,
even if it were some theoretical optimal solution or win win.
Speaker 1 (49:06):
Yeah. So I mean, so I guess, you know, we're
just in a very sticky spot and we'll have to
see how it plays out. I think Harvard is doing
its best, and you know, I wish that I could
give it, you know, a playbook, Harvard, this is what
you should be doing, but but I can't. So let's
just let's hold out hope. I think you are.
Speaker 3 (49:26):
I mean, there are real changes that like, yeah, Harvard should, and.
Speaker 1 (49:30):
I think Stephen and I think Stephen Pinker has advocated
for a lot of them. Like, as I said, I'm biased,
but I think that Harvard and Harvard should listen to
Steve Pinker. That's my advice. I think he's one of
the smartest people I know who actually knows a lot
about this and has very rational views and can advise
(49:51):
on this. It is almost eleven pm for me in
Hong Kong, so I think we're gonna rap it for
this week. The World to Years of Poker has already started.
You and I are not going to be out there
for another It's another basically two weeks, I think for
us week and a half. So let's wish all of
our Risky Business listeners who are already in Vegas for
the World Series of Poker good luck. We hope that
(50:13):
you all crush it at the tables and we're looking
forward to joining you soon. Let us know what you
think of the show. Reach out to us at Risky
Business at Pushkin dot fm. And by the way, if
you're a Pushkin Plus subscriber, we have some bonus content
for you that's coming up right after the credits.
Speaker 3 (50:32):
And if you're not subscribing yet, consider signing up for
just six ninety nine a month. What a nice price
you get access to all that premium content and ad
for listening across Pushkin's entire network of shows.
Speaker 1 (50:44):
Risky Business is hosted by me Maria Kannakova.
Speaker 3 (50:47):
And by me Nate Silver. The show is a co
production of Pushkin Industries and iHeartMedia. This episode was produced
by Isabelle Carter. Our associate producer is Sonia Gerwin. Sally
Helm is our editor, and our executive producer is Jacob Bolstein.
Speaker 2 (51:01):
Mixing by Sarah Bruger.
Speaker 1 (51:02):
Thanks so much for tuning in.
Pushkin. Welcome back to Risky Business, a show about making
better decisions. I'm Maria Kanakova.
Speaker 2 (00:31):
And I'm Nate Silver.
Speaker 3 (00:32):
Today on the show, we're gonna be talking about how
chat GPT plays poker, i e.
Speaker 2 (00:38):
Poorly.
Speaker 3 (00:38):
If chat gipt is getting ready for the World Series
of Poker like us, then it's got a lot of
homework to do.
Speaker 1 (00:44):
Yes. And after we talk about that, we'll talk about
Harvard two point zero aka the fact that foreign students
are now being potentially barred from attending Harvard University. Harvard
has sued the administration. Anyway, we'll be talking about what's
going on with that and what the implications are for
(01:06):
the future of the United States and kind of research development,
brain power and the US competitive edge. So for people
who don't subscribe to Silver Bulletin, which you absolutely should,
Nate and I both had poker posts this last week.
(01:27):
Mine was about cheating, was about chat GPT, but his
post this week was one of the funniest things I've
read in a while about his attempts to get chat
GPT to simulate a hand of cash game poker. And
one of the reasons I mean, I found this amusing
(01:48):
because I mean, first of all, it fused cards together
for an image. From the beginning to the end, it
was pretty spot on in terms of being spot on wrong.
But Nay, You're also someone who is constantly writing about
how good AI is at so many things, and so
it was funny to me to see it fall so
(02:08):
short of something that actually tests intelligence as opposed to
being able to kind of pull together a lot of
things and spit out an answer.
Speaker 3 (02:19):
There are some reasons why I'm just sid in testing
chatchipt on poker, but they kind of fall into two
big buckets. Right, What is it like poker played in
the context of a real hand or in this case,
a real text based simulation I guess of a hand.
It really does require quite a few skills, right. There
(02:40):
is the pure math part of it. What is the equilibrium,
the Nash equilibrium, the gto strategy that you're solving for.
There's also making adjustments for how other players play. There's
the conversation you're having, the physical reads that you're getting
their stuff, like knowing what the rules are right, which
might seem trivial, but we've yeah, chatchpt also.
Speaker 1 (03:03):
Does not know.
Speaker 2 (03:03):
It seems to be no, it is fun to look.
Speaker 3 (03:06):
I mean, I'm sure we've all had hands where we
like misread a board or I you know, I had
a big hand in the main event at the when
I play back in Florida back in April or whatever,
right where like a guy had like a twenty five
thousand dollars chip that was like almost the same color
as the felt, which is not good chipkit maintenance, by
the way, Seminal hard Rock. I like the WPT, but
(03:26):
that chip kit it's got to be improved, right, But yeah,
you're tired, you make mistakes like that. But and you
also have to have like a lot of short term memory,
which sounds trivial, but you you know, you want to
remember how you arrived at this current spot in the hand, right,
you have to keep crack of all these stack sizes,
(03:47):
which again seems trivial, but like it's a good test
of general intelligence of a certain type, especially a live
poker hand, like I'm asking.
Speaker 2 (03:56):
Chat ept to simulate.
Speaker 3 (03:58):
Right. The other reason is that like I don't think
engineers in Silicon Valley are trying to optimize it for poker.
And the reason that's important is because like, look, there
are benchmarks like math Olympiad problems that these LM's large
language models like CHATCHPT, Claude, et cetera compete on, right,
And they're like, well, we be bragging that we can now,
(04:20):
you know, beat all but the Nobel Prize mathematicians on
x percentage of math problems and things like that. And
there are a couple of issues with that, right. Like
one is that, like, clearly, if you train a transformer model,
a machine learning model on a particular type of problem,
like it can do fairly well or very well often, right,
(04:42):
Like poker is to a first approximation, and it's an
important approximation. Right, But if you have poker dedicated tools,
I wouldn't consider a solver and AI that's a technical
distinction I think might not be that important for our listeners.
But like, you know, if you want to use computers
to play very good poker, then they can play very
very good poker. Right. The question is like, can you
(05:04):
take a text based model and have this organic property
where intelligence emerges and converts from the data set towards
superintelligence without training it on poker specifically. And the answer,
at least as of last week when I did this,
is no, it fails miserably.
Speaker 1 (05:22):
Yeah, and I think it's actually important. One of the
things that came to my mind when I was reading
your piece was that poker has been a benchmark for
AI development well before LMS, Right, It has been kind
of the gold standard that teams all over the world
have been working on for decades because it is a
(05:46):
much more complex game in many senses than chess, then
even go, and it's a game where, you know, if
computers can actually manage to outthink human players on a
broader scale, that would mean something in terms of kind
of broader intelligence capabilities. And before LMS were developed, that
(06:08):
had not happened. Right, There had been computer programs, kind
of AI algorithms that have been able to beat heads
up one on one opponents in poker. But when it
came to full ring games, So for people who don't
know poker, that means basically, when you put it in
a rich environment with you know, six players, eight players
and you have a computer there, it was still not
(06:31):
quite there. And one of the you know, one of
the reasons why poker so interesting is it's not just math, right,
especially when you have multiple players. There's so many dynamics
and there's actually a very thorny problems for AI, which
is that if you program something to follow an algorithm
right to be GTO game theory optimal, then it can
(06:55):
be exploited by a human if the human figures out
the GTO strategy. Now, if it were another human, then
you'd adjust immediately right the moment the human adjusts, the
other human would adjust and kind of figure it out.
But if you're an a if you're a computer, and
you build in that adjustment parameter, what ends up happening
is that the model adjusts too much and too quickly,
(07:18):
and so that lacks kind of some of the nuance
and flexibility that marks that's kind of the hallmark of
the great poker players, right, kind of that ability to
into it when it's time to slightly deviate. And I mean,
it would be terrifying but amazing if a model could spontaneously,
(07:39):
like an LLM, could actually spontaneously figure that out and
so far, I mean, instead instead of doing that, Nate,
you know, your model decided that the person who won
the hand actually lost the hand, right, It just got
basic things completely wrong, and you just you know, obviously
we've talked about pe doom a lot on the show.
(08:00):
I think you and I are both worried about that.
But when I see this poker stuff and like, oh man.
Speaker 3 (08:05):
Yeah, I don't know how much to update. But look
my say in the post, I mean, like my appreciation
and expectations for large language models have increased a lot
over the past year. I had a pretty good experience
working with them when I was building the NCAA tournament
model that we host silver bulletin and just helpful kind
(08:27):
of Swiss army knives or like annoying data related tasks.
It's late at night, you forget the Stata command for something,
You're like, right, give me the right command and write
five lines of code, and like usually it works, and
it kind of now shows you like the chain of
reasoning and like what it's thinking. Right, And but you know,
people are saying that we're going to have like AI
(08:50):
revolutionizing all programming or you know, well, first of all,
the big claim is going to have AGI, meaning that
a machine. It's a little ambiguous when people say, does
this mean like chat GPT itself is AGI? Right, Well,
clearly not. It's very far from human being in some things, right,
(09:11):
Does that mean that a combination of transformer or machine
learning based technologies is or zooming out the definition of AI.
It's really AI still, right, so that you have chechiptiza
seventy percent of things well, and then you specialized applications
for twenty percent, and then there are ten percent where
you're not doing it very well. Like that would still
(09:31):
be a big achievement, very disruptive achievement. I'm not sure
it would kind of quite count as AGI. Like one
thing you could do is, like some of these new
models will say, okay, I can detect that you ask
for something that requires mathematical precisions. So let's take a
simpler example. Right, Let's say I want to take the
(09:53):
distance between pick two cities, Boise, Idaho and Tampa, Florida. Right,
that's pretty easy to calculate mathematically if you look up
the latlong coordinates for Boise and Tampa, and then there
are various formulas that.
Speaker 2 (10:09):
You can use.
Speaker 3 (10:10):
Right, So a pure data set where it's just crunching
texts might not find that answer because there's not necessarily
a lot of examples of like what's the distance between
Boise and Tampa? Right, Whereas you would have that for
like La and new York or something. However, you can
set up kind of agentic meaning agent ais where they're like, okay,
(10:31):
well this person asked a question that triggered a routine
of mine. Where now I'm going to go and I
know to search the internet for the coordinates or maybe
it's stored somewhere right of Boise and Tampa. And then
the formula. I know the formula and I can apply it, right,
so like you have the scaffolding. It's sometimes called a
different processes on top of one another. And I guess
that kind of I guess that counts as I mean,
it's kind of how humans solve problems, right, They're like, oh,
(10:54):
and now I have to go look something up, right,
It's still I think subtly different. So one implication I
think this has is artificial general intelligence versus super intelligence ASI,
sometimes called where there's like an explosion of intelligence just
from kind of mining text or other data sources. I
(11:17):
have actually become a little bit more skeptical of that,
or I think it's presumptive to assume it can find that.
I mean, like, let me give you another example, right,
like one I had an idea for either a silver
bulletin post or maybe even a risky business segment, right
where like I asked, chatchipt deep research, design a meal
(11:39):
with foods that have no clear precedent in other countries, right, huh?
And I thought, and the idea was that, like, you know,
at some point somebody invented pizza or spaghetti or whatever else, right,
or or I.
Speaker 1 (11:54):
Mean at some point somebody invented bread and figured out
that you can take flour and like bake shit with it.
It's amazing.
Speaker 3 (12:01):
Yes, you have to be able to buy these ingredientstead
of Trader Joe's or a Whole Foods. Right. I was
just going to like hire professional chef and have a
dinner part and ask how these creations went, right, And
it's like and all they recipes were like here's salmon
with miso paste, right, really creative and things like that, right,
and like it didn't have any ability to like extrapolate
(12:21):
beyond the data set. I'm sure these preparations are fine.
It was like their little chefy, meaning like things that
like have one too many ingredients, right, and like and
I don't, yeah, look so bad chefe.
Speaker 2 (12:35):
Yeah.
Speaker 3 (12:36):
And meanwhile, I've been like I've been interviewing candidates for
like a sports assistant position at Silver Bulletin, and one
of the questions is, how are you using AI? These
are very bright I want to say kids. A lot
of them are are recent college grads, you know, your
early twenties to mid thirties, basically right, and very technically
(12:56):
proficient people. And asked them how they're using a GI
and their experiences like kind of similar to mine. They're like, Yeah,
for a certain task, it's very helpful. For certain task,
it's somewhat helpful for certain tasks, not helpful at all.
I can get you in trouble, right, which again contrast
with the experience of people at the AI l apps
to say, yeah, it's going to like displace all programming
(13:20):
within a year or two.
Speaker 2 (13:21):
I mean, I know you should weigh in here.
Speaker 1 (13:23):
So I have an interesting kind of anecdote to add
to this that's not poker related, but that talks about
kind of some of this nuance and complexity and the
fact that you know that all of these lms still
are wanting in certain major respects. So I had the
chance last week to speak to the CEO of one
(13:46):
of the major AI companies. I won't name which company
it was, but one of the big ones. And someone
had asked him earlier, what was something that was surprising
about the way that this AI functioned, and he said,
you know, one of the mistakes that people is kind
(14:10):
of they trust it, And what we know is that
it's remarkably accurate ninety percent of the time, right, and
then ten percent of the time it isn't. But the
ten percent is getting harder and harder for humans to
spot because it doesn't make errors in a way that's
intuitive for people, right. It makes errors in different ways.
(14:30):
It's like computer errors AI errs, that's not the way
that the human mind makes errors. He's like, but like,
it's fine because we have really smart people who can
figure out where it's making errors and can work with
it and can kind of calibrate it and help. And
so my question, my follow up question, was, Okay, you
know what happens when the new generation that's being brought
(14:52):
up with this AI comes up and they've been educated
with it and they've been using it the whole time,
and they didn't necessarily get the same education that we got, right,
because they aren't the incentives are different, and they're not
getting the deep knowledge. They're not actually able to figure
out what are the programming errors because this person was
talking that, you know, some of the biggest potential of
(15:14):
AIS and like programming and biology and kind of those
those types of things, and I said, well, what if
that person doesn't have the neuroscience and the biology background
or the programming background. And what the CEO said was,
this is a major problem, and what we're hoping is
that we can develop even more advanced AI to help
and to help fix the problems that AI is causing.
(15:36):
Because he was like, yeah, this is a big problem
and we're not quite sure, like we're hoping for the best,
but we don't know. And the fact that this was
our conversation didn't exactly inspire me to heights of thinking
this is going to replace intelligence or become AGI or
what did you call, Nate the other ASI artificial super intelligent? Yeah,
(16:00):
artificial superintelligence. So it didn't inspire me that that was
going to happen, and instead it was like an uh
oh moment where like, what happened when the humans who
are kind of helping push it along and make it better?
When that generation, like when they age out and they're
new people out there who who lack that sort of
(16:22):
skill and expertise and who grew up with AI that
is ninety percent accurate or even ninety five percent accurate,
but they can't. It's harder and harder to spot that
five and ten percent. And that to me was actually, like,
it's a worrisome thing, and it's clearly worrisome to the
people who are developing these technologies as well. And so
(16:44):
that I think is something that dovetails with what you
were talking about here. What happens Nate if all of
a sudden, all poker players are training with chatchipt right,
and that's that's actually great for us, but not great
for the future of poker.
Speaker 3 (17:02):
And we'll be right back after this break. I use
large language models for tasks that I have strong domain
knowledge over and could do myself. But they're a labor
(17:23):
saving device. Often they say substantial labor, right, Like I
use them to copy edit articles I posted the newsletter,
and I you know, I'm a fairly proficient user of
the English language and like for programming, right Like you know,
if you're like using AI to like design a website
and some of the functionality breaks, right, that might be okay.
Speaker 2 (17:45):
You can patch it and fix it. Right, if I'm
doing things.
Speaker 3 (17:47):
Involving modeling, I'm trying to project the value of basketball
teams or football teams, or basketball players or football players. Right,
and there's a bug where all of a sudden, like
one of the worst players in the league is rated
as being very good. Right. I mean, and you see
this in poker, where like you know, AI is relying.
You get punished in poker for sloppy application of imprecise
(18:11):
application of heuristics. Like one of the things like the
AI did so at first, I had it simulate just
one hand, and it had like a backstore for each player.
It fucked that up, right, then like Oka, I'm gonna
use deep research now and have it simulate like an
orbit of eight hands.
Speaker 2 (18:26):
So that was a little better.
Speaker 3 (18:27):
It took more compute time, right, Some of the hands
were decent, but it you know, it didn't know how
to like read aboard correctly and keep track of stacks.
But like, but when it was over able to overcome
those errors, it also had poor strategy and it gave
various excuses. One thing it said is like, well, the
quality of text based content on poker on the internet
(18:49):
really sucks, Right, it's aware of that is it says
it's worse than go or chess, although when I talk
to people who know how AI's played chess, it's also
a disaster. Even something like wordle I heard from a user.
You know you think wordle is a word game and
it does very well, but like it can't quite figure
out the strike sure of the problem, right, it kind
(19:10):
of was learning a little bit more by by a
rote And to be clear, like this is a very
good strategy for like many types of problems. And also
if you were to say, okay, here's a bunch of
we bought data from PIO solver or gto wizard. These
are solvers if you don't know listeners or a database
of high stakes hands. Like if you trained it on
that and then had GPTs say okay, poker question, I'm
(19:33):
going to call them especially trained database, then it would
do well. And maybe if people are criticizing its for
on poker, then the labs will start to do that. Right,
but it's still kind of like slightly cheating from the
standpoint of super intelligence. Also, it says like when you
asked me to do a whole bunch of things at once, right,
So I told it, give me characters, give me eight
characters who have like they were all kind of like
(19:55):
ethnic stereotypes, right, it was.
Speaker 1 (19:57):
They were very funny.
Speaker 3 (20:01):
It was kind of yeah and great. It was kind
of at the one hand, so half the players were women.
That was very feminist of it, right, But then they're
all ethnic stereotypes of different kinds, right, and so it
was kind of both woken and unwoke, right the opposite. Yeah,
it was kind of good about like matching players playing
styles with with their actions, but like, but it couldn't
(20:24):
keep track of stack size. It's like, yeah, keep track
of stacks. I have to store a bunch of stuff
in memory, and then when you ask me to do
a whole bunch of things at once, right, and like
you know, and even including like so in the hand
I show a sylver bulletin it like it misreads the board,
doesn't recognize that a pair of nines has a higher
two pair. If you ask it that out right, then
(20:46):
it thinks about it more and gets the question right, right,
But like it loses track of things in the context
of these scaffolding situations where it has to keep track
of a bunch of things at once and it doesn't
know it doesn't know what to prioritize right. Like you know,
you can make a lot of mistakes in poker if
you don't know which hand be twitch hand. That's a
more elementary mistake than anything else.
Speaker 1 (21:08):
Absolutely, So there are a few things that stand out
to me here.
Speaker 3 (21:11):
One.
Speaker 1 (21:12):
I mean, it's a computer, right, like we humans have
working memory capacity problems. It should be able to keep
all these things like this is what it's good at.
But the problem that you're kind of that you're illustrating
is it doesn't know how to prioritize right, and that
that is actually that is a major problem. And also
(21:32):
when we keep saying thinking. But one of the first things,
so when I was just learning poker, one of the
first lessons I had with Phil Galfon, who is one
of the people who kind of I worked with and
who taught me a lot. I remember very early on
he said, you know, I can give you a bunch
of charts and a bunch of outputs, and you can
memorize them and you'll be a very decent player very quickly.
(21:55):
Like I know, I can give you this shit, you
can memorize it, you can spit it back at me.
You'll be fine, He's like, but I don't want you
to do that, because that's going to make you a
fine poker player, and you'll do fine in the short run,
but you'll never be a great poker player because you
have no idea why you're doing it. You're not actually thinking,
you don't understand. You've just done a rote memorization, which
(22:16):
is might make you more money in the short term,
but in the long term is actually going to be
detrimental to you because you're going to do stuff unthinkingly
because you've memorized it.
Speaker 3 (22:25):
He said.
Speaker 1 (22:25):
What I want you to do is instead think through
things and figure out, Okay, why right? For every single play,
why am I doing this? Why would I play this
hand and not this hand? Why would I raise this
type of hand and not this type of hand? Why?
Speaker 3 (22:41):
Why? Why? Why? Why?
Speaker 1 (22:43):
And that's something that to this day has stayed with
me because it's such an important thing to remember when
you're making a decision. Right, And this doesn't even have
to be poker, It can be any sort of decision.
Why am I choosing this action? Why is it better
than all of the other actions? Right? And if you
even if you feed you know, piosalvereign gto Wizard outputs
(23:05):
two lllms, they'll be doing some rote of memorization. You know,
at this point they're not understanding the why, and so
that will lead them still to make big mistakes, which
happens when you just learn solver outputs, no matter how
sophisticated those outputs might be and how correct they might
(23:26):
be in one specific hand, in one specific spot, if
you don't understand the why, you're either going to overgeneralize
it right or misapply it like you're going to screw
it up because you don't understand the underlying reasoning. And
there's the difference between looking like you're thinking and actually thinking.
(23:46):
And yeah, you know, as a human you have other pitfalls,
and even if you're you know, thinking through things, you
can mess up, So you mess up in different ways.
But I think that's a really important distinction and probably
one of the reasons why AGI is not as close
like poker. Actually illustrates in a very practical way, why
(24:08):
the notion that you can just have a lot of
data and all of a sudden have this you know,
flurry of insight might not be as I guess, as
intuitive as one might think.
Speaker 3 (24:19):
When one thing, Chatchip you told me when I asked
to like audit itself, is like, well, poker's very difficult
because it's adversarial. You know, understand that there's some game
theory there, but it's adversarial, and like there's no one
I mean, there is like I guess some superstructure of
like a solution for all poker hands if you get
(24:39):
very zoomed out, right, but like but like subtle things
when you have to calculate like an adversarial equilibrium on
the fly basically, and it's that's very difficult.
Speaker 1 (24:52):
Yeah, I mean think about not even poker, but think
about trying to do a game theory solution for multiple players.
Speaker 3 (24:59):
Right.
Speaker 1 (24:59):
It's hard enough trying to do a payoff matrix for
two players when you're really trying to think through it
and you're trying to think of all of the different
payoffs and figuring out, you know, how to exactly do
you wait them, how exactly do you structure that. Now
when it's three players, four players, I mean, it's incredibly
difficult to do that accurately. And so even if we
(25:19):
just zoom out from poker, like this is just an
incredible a very tough problem, even if you understand game theory.
Plus I mean, I think that anyone who has used solvers,
and anyone who's talked about this understands anyone who doesn't
even play poker but has worked with algorithms, the common
saying garbage in, garbage out is absolutely accurate.
Speaker 3 (25:39):
Right.
Speaker 1 (25:40):
The solver works based on what ranges of hands you
as a human put in there, right, and what responses
you allow or don't allow, right, And yes, it will
come up with an equilibrium strategy. But if you were wrong, right,
if there's a player who's playing a totally different range,
if there's a player who's playing a totally different strategy,
(26:02):
all of a sudden, your solution means nothing. And as
a human you can figure that out, and you can
kind of make adjustments from baseline, right, if you understand
what the baseline strategy as you can adjust as an LM,
as an AI who is learning that but doesn't kind
of have that nuanced experience at least at this point.
I don't pretend to know what AI is going to
(26:24):
be capable of in you know, in five years and
ten years, but at least at this point they're not
capable of making those sorts of nuanced extrapolations and figuring out, Okay,
what were my inputs accurate, right, or were my inputs
not quite accurate.
Speaker 3 (26:43):
Look, one thing human beings are good at is that
human beings are relatively good estimators, and chatchipt can be
sometimes if it's like, Okay, take all this text on
the web and kind of give me the average of that,
Like there are some applications where it's pretty good. But like,
you know, the other day, I was getting getting a
drink with my partner and a guy who looked like
(27:05):
he was homeless comes in and like hands like the
hosts slash barked like a note saying call nine to
one one or something like that, right, and you kind
of have to make this judgment call about like is
this like a paranoid schizophrenic or is someone actually in danger?
And like I think was pretty clear. I mean they're
(27:26):
probably you know, two bartenders and eight customers clear to everybody.
Like this guy I think is not like acutely in
danger or anything. But I'd never quite experienced situation like
that before. And like the fact that we can use
like our general intelligence about like human behavior Yeah, Like
I think everyone made the right decision that we're not
going to call the police either to rat on him
(27:47):
or to say he was in danger. And like you know,
and that kind of thing is is is hard for
language yells to do, you know.
Speaker 1 (27:57):
No, I mean I think in general, like this just
illustrates a really good point, which is that humans, even
not the smartest humans, are much smarter and much more
capable in very basic ways that we take for granted
than a lot of kind of super intelligences. Right, Like
it took forever for a robot to be able to
(28:18):
pick up an egg, right, It's something that we don't
even think about, but this was like a robotics problem
that was absolutely unsolvable, Like how do you get a
robot to pick up an egg without without breaking it?
And we do it without just without thinking about it.
And we make judgments all the time, just silly judgments
(28:38):
that we don't think twice about that are so easy
for the human mind, but that are that we don't
even realize we're making right things about safety, things about
just perceptions of the world. So I think that you know,
humans are much smarter than than we think in like
in dumb ways, if that makes sense, Like in waste
(29:00):
that seemed like they're not hard problems, but those are
actually some of the hardest problems to solve.
Speaker 2 (29:05):
I just think of.
Speaker 3 (29:05):
There being four levels, right, Like one more AI performance
better than any human, two where AI performs better than
like all but expert level humans, Three where AI gives
kind of a passable substitute performance but like you know,
not quite professional high grade level, and four where it's
just inept right to the point of being comically inept.
(29:28):
And like, you know, my heuristic is that like within
a few years we might have roughly an even divide
between those four and I'm counting, by the way, tasks
that involve manipulting the physical environment, which I think AI
will mostly be pretty bad at. Right, I don't know
how much I should extrapolate from the poker example. It
was just so incongruent with these predictions of like imminent AGI, right,
(29:54):
where a year ago would have said, okay, yeah, of
course suck at poker. Right, it's not really trained on
this and it's a hard problem, and ha ha ha, Right,
But like if you want to have these complicated, like structured,
nested tasks, like I'm much less worried now in periods
of oh, let's say five years of AI like being
(30:15):
able to build like atiscal model to like forecast elections
or the NFL or whatever else, because that like just
requires like a superstructure of lots of little tasks, all
of which are fuck upable, And if you get the
superstructure wrong too, then you're kind of just drawing dead.
To use a poker term, right if any if you
(30:35):
have to chain together twenty steps and any step, there's
a ninety percent chance you get it right. Well, point
nine to the twentieth power means you're almost.
Speaker 2 (30:42):
Sure to fuck something up.
Speaker 1 (30:45):
Yeah, fuck upable is a really great word to describe this.
Nate to kind of to sum this up, you did
ask chat GBT why it was bad at poker? What
did it tell you?
Speaker 3 (30:55):
Yeah? No, I mean it's said various things. It said, Look,
you stressed me out by having me having me have
to make up these players and things like that.
Speaker 1 (31:03):
Trust it out, You trusted.
Speaker 3 (31:06):
It said the data that it trained on on the
internet is which seems realistic.
Speaker 1 (31:11):
Yeah, it's a data problem. It's a yeah, it's not me.
Speaker 2 (31:14):
It's you. It's it's complicated.
Speaker 3 (31:18):
Inform about like how like it doesn't realize how important
stack sizes are because it's just a bunch of tokens.
Like all the input you give it is generated into tokens, right,
and like the word the is not very important. The
fact that Maria has fifty two thousand dollars in chips
is very important, right, And it doesn't know how to
distinguish those from its transformer architecture. So like, look, it
(31:41):
is good at like it is good at verbal reasoning.
Like again, I I, you know we are talking about
we don't other segments where we're like amazed by AI progress.
Like it's very good at verbal reasoning, or at least
faking that. But in ways I if it's faking, it's
doing a pretty good job. Right, Like it's very good
at verbal stuff for the most part. Right, And AI
(32:02):
is traded on poker, are very good at mimicking Selver solutions,
right and like and so like and machine learning can
get you a lot of ways when you have good data,
but like when you don't have the data in the
training set and it's not seeming to extrapolate very well,
and then these complex tasks that require more and more
compute like, I have not been impressed by deep research,
(32:23):
which is where it goes away and says I'm going
to perform several tasks for you that require deeper thought, right,
and then it takes fifteen minutes you come back and like,
are you fuck this up? Right? And like, anyway, so
I think it has affected my priors a little bit.
Speaker 1 (32:36):
Yeah, that's that's really interesting. And I love, by the way, Nate,
that you know, even though that this is an AI
which is supposed to kind of be better than humans
in so many ways, that use such human excuses for
why it fucked up. You know, you stressed me out,
you gave me that data. It's not my fault.
Speaker 2 (32:53):
This is hard.
Speaker 1 (32:55):
And on that note, let's let's take a break and
talk a little bit about Harvard and the other type
of intelligence, human intelligence. Nay, We've talked on the show
(33:16):
a few times now about kind of the crusade that
Donald Trump seems to have against higher education, specifically the
Ivy League and even more specifically Harvard University. Last time
we spoke was when basically the administration had threatened to
pull funding if Harvard didn't comply with a certain set
(33:37):
of demands. Harvard said fuck you and sued and yeah,
and tried to go that route, and the funding was frozen.
And then the administration last week, so we're taping this
on Wednesday, the twenty eighth of May. So last week,
(33:58):
the administration decided that it wanted to punish Harvard even
more because you know, they didn't like the fact that
Harvard wasn't complying, that it was being defiant, that it
was taking them to court, and so said, hey, Harvard,
you can no longer have foreign students. Anyone who has
visa's effective immediately. Those are going to be revoked. So
(34:19):
Harvard's class of twenty twenty five that's graduating next week
is going to be your last basically foreign student class,
and we're not going to be granting visas anymore to
those students. Harvard has sued again, so now we have
a second set of lawsuits. The initial court cases have
frozen that. But then after that happened in the last
(34:43):
few days, the administration said Marco Rubio actually said that
we are going to freeze interviews for all foreign student visas,
not just Harvard. So they not only you know, went
after Harvard, but doubled down and said, you know what,
you can sue us and you can freeze this. But
if we don't grant the interviews, last laugh is with us.
Speaker 3 (35:06):
Yeah.
Speaker 2 (35:07):
Were they likely to win the lawsuit or are the
likely to?
Speaker 3 (35:11):
Yeah?
Speaker 1 (35:11):
I think so. I think that Harvard is likely to
win the lawsuit. Yes, However, from the lee I'm not
a lawyer, from the legal analysis that I have seen,
they're likely to win the lawsuit. But Nate, let's remember
that that requires first of all, a lot of times, right,
because how many times can we appeal this? And ultimately, well,
if it goes to the Supreme Court, it ends up
(35:34):
being these days less of a legal issue and more
of a political issue.
Speaker 3 (35:39):
Look, I don't think the Supreme Court is likely to
be very sympathetic to Trump on this type of issue,
in part because, like you know, clearly they are somewhat
transparent about like they are not doing it for like
some security concerning and they're doing it because they don't
like some of the speech that Harvard is making. But
it's almost certainly First Amendment protect.
Speaker 1 (35:59):
Yeah, exactly, and I actually think that the So I
don't know if you read Stephen Pinker's op ed in
the New York Times, I don't remember what the op
ed was called, but he termed this Harvard derangement syndrome, right,
kind of playing off of Trump arrangement syndrome and full disclosure.
Steve Pinker was my undergrad advisor, so someone I know, well,
(36:19):
someone i'm you know, I'm still close with, so I'm
very sympathetic. But I thought it was an excellent way
of framing what's going on. And he is someone who
has attacked Harvard. He's a tenured professor at Harvard, but
he's attacked Harvard many, many times for not standing up
for free speech, for not protecting conservative viewpoints, for being
(36:41):
a little bit too woke. Right. He has been one
of the main people who said, hey, like, we have
an issue here. But this his op ed was, this
ain't the way, right, this is not the way to
solve this problem. And you're actually attacking the students who
are some of my most critical, thinking, least woke people
who are on campus.
Speaker 3 (37:01):
But yeah, look, I mean I kind of want to
zoom out and say, like, what is the equilibrium here,
Like what is a Trump administration trying to achieve? What
is Harvard trying to achieve? Like I support maybe some
listeners will get pissed off, right, you know, I think
clearly some of these colleges are not complying with like
the spirit and maybe the letter of the Supreme Court's
(37:24):
affirmative Action decisions, right, and so like, I wouldn't mind
if like the Trump administration's like aggressively enforcing the law
on those or things like diversity statements, which are enforced
political speech. Although Harvard's a private organization, so it's a
little bit more complicated, right, But like I you know,
I don't it's such a blunt tool to like target
(37:46):
foreign students. And even if you hate Harvard, you know,
as a patriotic American, I want the best and brightest
from all around the world to come here and contribute
to our economy and pay our taxes and everything else, right,
And like it's just like purely destroying the value of
these very bright students from helping our economy.
Speaker 1 (38:04):
Yes, it's destroying our intellectual capital. So, by the way,
when we're taping this right now, I'm actually in Hongo,
and I just found out today that several universities in
Hong Kong have offered blanket admissions to all students who
can show that they were admitted to Harvard and can
no longer go because they are a foreign student. They said,
you don't even have to apply, you can just come here,
(38:24):
and these are some of the best universities in Hong Kong.
And that's super smart, right, Like, universities all over the
world should be doing stuff like that right now, because
what the administration is doing is kind of bankrupting at
the future of kind of intellectual development in the country
by saying, oh, foreign students, you know, we've we've just
frozen all these applications. So to me, like that's super smart, right,
(38:47):
Like that is the we've we talked about this several
months ago, Nate, Right, what can China do to capitalize
on the moment it's doing it. It's being like, come here,
like you don't even have to apply, you don't have
to do anything to show us the letter that showed
you were accepted and welcome. Come.
Speaker 3 (39:02):
Yeah. I went to London School of Economics for my
junior year and if I were them, i'd be doing
cart wheels. And they always got a lot of foreign students.
Speaker 1 (39:10):
Right absolutely, I mean I think everyone should be doing
this right now, and I just do not see I
mean a lot of these things. At this point, it
just derangement syndrome seems right, because there does not seem
to be any strategic value to this. There's not any
any value in terms of trying to get at the
types of problems that the Trump administration says it wants
(39:30):
to get at, because it's not even addressing that right,
Like at the originally anti Semitism was the pretext for this,
and this is just like so far divorced from it.
Everyone is having their visas revoked or not granted, or
their interview is not even granted. So it's just, you know,
the pretense is now crumbling. We always knew it was
just a pretense, but at this point, it's just not
(39:51):
accomplishing anything other than to further could of create this
absolute chasm in higher education and future brain power, future
research ability, future creativity of the United States basically mortgaging
its future right saying that we were going to give
up one of the biggest, if not the biggest edge
(40:13):
that the United States has had historically, which is kind
of this brain power creativity place where people can can
develop and find support for for for their ideas, and
now that is not possible and if this actually stands, right,
(40:35):
if foreign students are not allowed into US universities, because
right now, like I said, this interview process being paused
for everyone, not just Harvard. I mean, that is going
to be just detrimental in so many respects to what
happens here.
Speaker 3 (40:53):
Yeah, the next So Harvard is not a very sympathetic
case for reasons that are partly they're full right, but
like you know, so far, the Trump admin has been
pretty smart about kind of which schools they they pick on.
I mean, I don't know what their in game is
here or what Harvard's is really you know, JD. Vans
had it tweet. I don't know where he was like,
he's oddly transparent sometimes about his thinking, and he's like,
(41:16):
well look just like, look, these colleges aren't serious about
complying with the law citing like the students for fair admissions.
I think it's called which was the affirmative action decision,
which I do think they've kind of not been serious
about complying with in some cases case by case basis,
and so like, unless we kind of show real muscle,
then what can we do. We have to fight them
because they're in the wrong, and so we're overreaching. It
(41:38):
is a subtext of that, right, which is like maybe
not crazy and they have a real leader because Harvard
isn't that sympathetic perceptions of higher education that's gone way down,
so it's like harder to marshall public opinion on the
side of this, right, and like and like, you know,
I saw some Harvard professors on Twitter. Ever we can
(41:59):
be like this is the greatest threat to American I'm like,
please don't say that, because it's it's it's maybe more
of a threat to American competitoris and not sure it's
a threat to democracy per se.
Speaker 1 (42:09):
Right, No, I think there are bigger threats to democracy.
Speaker 3 (42:13):
But like, but what's Harvard's end game? Like I think, ok,
if they could go back and tone down some of
the things say down for the past ten years, and
I think they maybe would, but like now, I mean,
if they back down, then they look weak and that
also might hurt their recruitment own nothing, they'd have real problems.
But like, what if you're the president of Heart What's
(42:34):
name Alan Garber? Is that right? Yep? If you're Alan Garber, Maria,
what do you do right now? And you know Harvard?
Speaker 1 (42:40):
Yeah, yeah, I mean, you're in a horrible position. I
think you're doing what you can, which is trying to
stand up to the Trump administration. So Alan Garber is
trying to actually walk a very fine line. He's trying
to stand up to Trump, and he's you know, spearheading
these lawsuits, and he is leading committees into the claims
(43:03):
of anti Semitism and kind of and discrimination on campus,
the anti wokeness. So he is actually trying to address
all of these things in different ways. And I have
no idea, you know, how successful that will end up being,
but it is. It's a it's a tough you know,
he's been put in a really tough spot because obviously
(43:24):
he hasn't been president for long. The last president departed
on lessons tellar circumstances, as did a number of presidents
of Ivy League institutions, and you know, you're picking up
the presidency at a very difficult point in time where, yeah,
(43:45):
you have to acknowledge that Harvard has made mistakes, right,
not just Harvard, a lot of universities I think were
trying to be I don't even want to use the
word woke, because I don't think that's accurate. Here but
we're becoming, you know, places where people were afraid of
voicing their minds. And I wrote, I think almost a
(44:07):
decade ago, I wrote a piece for The New Yorker
about basically, you know, the problems in academia that come
from not having enough conservatives right that it actually has
trickle down effects in research and the types of things
that are being researched in the findings that are allowed
to be voiced, and that it can be incredibly problematic.
And so I think you need to acknowledge that, and
(44:30):
those are things that need to be rectified. At the
same time, you are currently fighting this battle against you know,
Donald Trump, who's trying to destroy higher education in the
United States, and so that is a you know, overriding priority,
which doesn't dismiss all of the things that have gone
wrong and all of the things that are wrong with academia,
but it's just a it's a kind of this fine
(44:52):
balance where you're like, yeah, I know I'm not sympathetic
because I've fucked up and I've done all of these things.
And yet right now I'm one of the people best
positioned to fight this, and so let's just try to
figure out how to prioritize this and how to not
spread ourselves tooth in so that we can actually protect
higher education and then start to remedy the problems that
(45:14):
we have internally, because I do think those are problems
that need to be remedied, right, Like, you can't just
be like, well, now we're all united against Trump, so
let's forget we ever did anything wrong. I think that
all of these things you remember, Nate, like less than
you know. Like six months ago, I was railing against
Harvard because I was really pissed at the way they
were reacting to a lot of things. Now I think
(45:37):
now I'm supporting it because they're being attacked. It's the
classic psychology thing, right where you can actually unite a
lot of people who who criticize you when you attack, right,
all of a sudden, these various factions, when they feel
these life threatening attacks from the outside, all of a sudden,
these factions start uniting. And I think that's what's happening
at Harvard right now, as it should be. But we
(46:00):
still have to realize that, you know, there are all
of these different issues and it's a really I mean,
it's a shit show. That's a technical term.
Speaker 2 (46:08):
Nate yeah, Look, I'm not afraid to use a term woke.
Speaker 3 (46:12):
I mean, you know, critical race theory and intersectionality, all
these things kind of come from an extremely academic context, right,
And Look, the thing I'm probably most concerned about is
the fact that I think the quality of academic research
is often has lots of problems, but like political bias
(46:32):
in predictable directions is among those problems. I'm trying to
think of a politically experienced solution that the Trump administration
might like and that people would find tolerable. On the
Harvard side, Right, we'll collect data on the political affiliation
of our students and faculty. We will make concerted efforts
to I'm not going to try to frame this right
(46:53):
conservati efforts like higher conservative scholars. Right, maybe you have
like you know, if you have an ethnic studies department,
you have a conservative stay.
Speaker 1 (47:00):
No, I mean that isn't that you know exactly what
people have been trying to reverse by kind of with
the affirmative action rulings, right, that you have quotos for
certain types of people. So I think that the better
way to do it is basically having blind hiring, just
like you have blind admissions. You are willing. You just
don't you don't ask any of that, and you don't
(47:22):
disqualify people because they have conservative leanings, right, and that
it's not even it's not a thing because you don't
really care, Like I don't care if my uh, you know,
neuroscience professor, Well maybe not neuroscience as insofar as I
study psychology, but if my astrophysics professor is, you know,
(47:43):
what their political views are, as long as they correctly,
you know, as long as they're one of the best
astrophysicists in the world. So so I don't think that
that's you know, that what you're proposing is affirmative action
for politics.
Speaker 3 (47:57):
I'm just saying, like it's slightly stupid, but like maybe
maybe jd.
Speaker 2 (48:00):
Vance would like it.
Speaker 1 (48:03):
But but you're saying, like do something that he would like.
Columbia tried to do everything and had all of their
funding frozen anyway, right, So like this is the problem
that like actually doing trying to appease is not the
correct play here. I think that's actually the completely wrong
game theoretical way to respond here. That's because it's not working.
Speaker 3 (48:24):
And that's a meta thing across everything that the White
House does. You see it playing out Ontari's too, where
on the one hand, they reverse course and undercut themselves
all the time, right, you know, and they're sloppy enough
where they probably maybe if they're doing this right, they
could have had more likelihood of success in the courts
than they have. Right, on either hand, they don't necessarily
(48:46):
hold up their quid pro quote. This is kind of
very transactional and explicit, right, So they're not even like
presenting clear enough information to make themselves easy to bargain with,
even if it were some theoretical optimal solution or win win.
Speaker 1 (49:06):
Yeah. So I mean, so I guess, you know, we're
just in a very sticky spot and we'll have to
see how it plays out. I think Harvard is doing
its best, and you know, I wish that I could
give it, you know, a playbook, Harvard, this is what
you should be doing, but but I can't. So let's
just let's hold out hope. I think you are.
Speaker 3 (49:26):
I mean, there are real changes that like, yeah, Harvard should, and.
Speaker 1 (49:30):
I think Stephen and I think Stephen Pinker has advocated
for a lot of them. Like, as I said, I'm biased,
but I think that Harvard and Harvard should listen to
Steve Pinker. That's my advice. I think he's one of
the smartest people I know who actually knows a lot
about this and has very rational views and can advise
(49:51):
on this. It is almost eleven pm for me in
Hong Kong, so I think we're gonna rap it for
this week. The World to Years of Poker has already started.
You and I are not going to be out there
for another It's another basically two weeks, I think for
us week and a half. So let's wish all of
our Risky Business listeners who are already in Vegas for
the World Series of Poker good luck. We hope that
(50:13):
you all crush it at the tables and we're looking
forward to joining you soon. Let us know what you
think of the show. Reach out to us at Risky
Business at Pushkin dot fm. And by the way, if
you're a Pushkin Plus subscriber, we have some bonus content
for you that's coming up right after the credits.
Speaker 3 (50:32):
And if you're not subscribing yet, consider signing up for
just six ninety nine a month. What a nice price
you get access to all that premium content and ad
for listening across Pushkin's entire network of shows.
Speaker 1 (50:44):
Risky Business is hosted by me Maria Kannakova.
Speaker 3 (50:47):
And by me Nate Silver. The show is a co
production of Pushkin Industries and iHeartMedia. This episode was produced
by Isabelle Carter. Our associate producer is Sonia Gerwin. Sally
Helm is our editor, and our executive producer is Jacob Bolstein.
Speaker 2 (51:01):
Mixing by Sarah Bruger.
Speaker 1 (51:02):
Thanks so much for tuning in.
Hosts And Creators
Maria Konnikova
Nate Silver
Popular Podcasts
Boysober
Have you ever wondered what life might be like if you stopped worrying about being wanted, and focused on understanding what you actually want? That was the question Hope Woodard asked herself after a string of situationships inspired her to take a break from sex and dating. She went "boysober," a personal concept that sparked a global movement among women looking to prioritize themselves over men. Now, Hope is looking to expand the ways we explore our relationship to relationships. Taking a bold, unfiltered look into modern love, romance, and self-discovery, Boysober will dive into messy stories about dating, sex, love, friendship, and breaking generational patterns—all with humor, vulnerability, and a fresh perspective.
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com