April 10, 2019 • 28 mins
In this episode, Matt Taylor, Numenta Open Source Community Manager, interviews Numenta Co-founder Jeff Hawkins about some of his latest research ideas. In particular, they discuss the thalamus.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Jeff (00:00):
One of the things we knew is that sequences in the brain
have to be able to do is youhave to be able to speed them
up, slow them down.
Matt (00:04):
Right.
Yeah.
Jeff (00:05):
Like I can play a melody back faster.
I can recognize it faster.
I can speak faster and slower.
So there's all these-
Matt (00:09):
And it's not different patterns.
It's the same pattern.
Jeff (00:11):
Yeah.
It's the same pattern.
But you can speed up and slow itdown and so, um, I, there had to
be a mechanism for that in thebrain and if you think about it,
that mechanism has to apply notto one column in the cortex, it
would apply to like all thecolumns that are doing, that are
doing speech, something likethat.
Christy (00:35):
You’re listening to Numenta On Intelligence, our
podcast series on howintelligence works in the brain
and how to implement it innon-biological systems.
In this episode, we’re going totake you back to where the
podcast started– a conversationwith our co-founder Jeff
Hawkins.
In fact, for the next fewepisodes, we’re going to try
something a little differentthan the previous few episodes,
which featured Interviews withNeuroscientists.
Matt Taylor, our Open SourceCommunity manager, will be
bringing you conversations withJeff Hawkins, or Subutai Ahmad,
our VP of Research, on a varietyof topics.
These could include anythingfrom Jeff’s latest research
developments that haven’t yetbeen documented, to Subutai’s
updates on applying our theoriesto today’s machine learning
systems.
These will be casualconversations, the kind that
happen every day here atNumenta, and we want to share
them with you, our listeners.
Some of them may get prettytechnical, but we’ll provide
links to resources for furtherreading.
I hope you enjoy theseconversation from Numenta On
Intelligence.
Matt (01:34):
So I'm here with Jeff Hawkins in his office.
It's early in the morning hereat Numenta.
There's just me and you andwe're going to talk about some
speculative thoughts andthinking you have about the
thalamus, right?
Jeff (01:45):
Yeah.
Yeah.
So, uh, we chosen, I've chosen,uh, to talk about the Thalamus
now.
I'll tell you what it is and whywe think it's important.
And then some new ideas we haveabout what it might be doing.
Matt (01:57):
I think people will be interested cause we, we voted,
we usually just talk aboutneocortex.
So it's a little refreshing togo to somewhere else in the
brain.
Jeff (02:04):
Yeah.
So let's just paint a picture inyour mind.
You know, the neocortex is, islike a big sheet, like a big
napkin wrinkled up on top ofyour brain.
And it's only a couple ofmillimeters thick.
The thalamus, it's right in thecenter of the brain and it's two
parts, like everything in thebrain.
And there's like maybe like theshape of two eggs, two small
eggs.
Matt (02:22):
Right.
One on either end, eitherhemisphere.
Right?
Jeff (02:24):
So we, you know, that's everything in the brain has got
two parts.
We don't usually focus on that,but they're doing the same
thing, just divided in half.
So the thalamus is right in themiddle and um, it has a very,
uh, unique relationship to theneocortex.
In fact, myself and some otherpeople studying the thalamus
really don't think you couldseparate them.
Um, and the reason is, is thatwhen information goes to the
(02:46):
neocortex, from your eyes, yourskin or your, or your ears, it
always goes through thethalamus, always.
Matt (02:53):
Sensory information...
Jeff (02:53):
All sensory information, pretty much, pretty much almost
any sensory information.
And so the object nerve, itdoesn't go from the back of the
retina, it doesn't go straightto the cortex.
It goes to the thalamus and thethalamus then goes to the
cortex.
Matt (03:05):
Right.
Jeff (03:06):
And when two regions in the neocortex project to each
other, you know, region A andregion B or v1, v2, there's they
primarily do that through thethalamus again.
So v1 projects to the thalamusand then the thalamus predicts
to v2 that there are directconnections between these
cortical regions, but there'salways one that goes through the
thalamus.
And since all information goesthrough the thalamus, it used to
(03:29):
be thought that the thalamus islike the gateway to the cortex.
Like when you get through thisgateway, you're in.
But now it's believed thateverything that goes between
cortical region, the corticalregion goes through the
Thalamus.
And that way you could think ofit as a part of the neocortex.
I kind of think of it like anextra layer of cells or extra
set of tissue that has been,that's been consolidated down
(03:53):
the center of the brain.
But it's almost, it's sointimately connected with the
neocortex.
You can't really separate themout.
Matt (03:58):
But it is an older part of the brain, right?
Jeff (04:01):
Ah, I don't know that's true actually, Matt.
I, you know, it's in the sensethat it's not neocortex.
We often tend to think abouteverything that's not the
neocortex is the older part ofthe brain.
I don't know if the thalamusexisted prior to the neocortex.
I don't know that.
Yeah.
So, um, it probably did in someform, but really I think we, if
you really want to understandhow the neocortex works,
(04:22):
ultimately you have tounderstand how the, what the
thalamus is doing.
Matt (04:25):
There are no mammals that we can find that just have a
thalamus and no neocortex
Jeff (04:29):
No mammals, all mammals have a neocortex of the question
is, or the non mammals that havea thalamus that don't have an
neocortex.
I don't know the answer thatone, but we can, we can pretty
much say right now the neocortexis intimate and required, and
the thalamus, is required tohave a neocortex.
So you can't really separate thetwo out.
Matt (04:48):
Well definitely the projections show that
Jeff (04:50):
That's right.
The projections show that,physically it's separated.
But the projections are veryintimate and um, and so we've
always known that any theory ofneocortex is going to have to
explain what the thalamus isdoing.
Um, and the thalamus itself isnot super complicated, but it's
not super simple either.
And I'm going to break it intotwo broad categories.
(05:10):
There's one broad category we'vejust been talking about, which
are called relay cells
Matt (05:16):
Relay cells.
Jeff (05:17):
Yeah, so like an axon comes from your eyeball and
Matt (05:20):
Oh it's synapses in the thalamus...
Jeff (05:22):
It makes a connection to one of these relay cells in the
thalamus and the relay cells inthe thalamus goes to the
neocortex and it literally lookslike one to one.
Matt (05:30):
Wow.
So just one, one stop and then
Jeff (05:32):
That's right and not only one stop it.
It's, there's very little con-,there's no convergence in the
thalamus.
It's like it's literally likeyou took a long wire and you cut
it and cut it in half and thenconnect it back together.
Matt (05:44):
So why do it?
Jeff (05:44):
That's right.
Right.
That's a good question.
So the call of the relay is alittle bit misleading because we
know it must be, there's noreason you're not going to have
this to do nothing.
Right.
But that's kind of what it lookslike.
They can say, this will show inmany situations, it's one to one
correspondence.
A spike comes in and spike comesout and there's a, the topology
of the arrangement of the cellsin the retina are preserved in
(06:04):
the Thalamus and they project tothe cortex and they're preserved
in the cortex.
So, one thing is why are thererelay cells?
The second thing, there's awhole bunch of other cells and
by the way it was a relay cells,a group of relay cells that go
between the retina and V1-that's called LGN lateral
Janeka.
No, cause that's just a bunch ofcells in the Thalamus and the
separate ones that go betweenall the other regions.
(06:25):
So these are dedicated cells forthese units.
Matt (06:26):
Right.
They're localized to
Jeff (06:29):
to whatever that projection is.
There's another set of cells inthe thalamus which are much more
diffuse.
They project to the cortex andthey project very diffusely.
So these cells, uh, they go bydifferent names.
I prefer the one that sometimesthey're called matrix cells.
I'm not referring to the movie,but referring to the, uh, the
fact that they're sort ofinterstitial to these other,
(06:51):
they're sort of in the matrix ofthe thalamus.
Matt (06:53):
When you say diffuse, just to define that term and the
connection is diffuse that'slike a direct...
Jeff (06:58):
Yeah.
So let's say a relay cell fromthe thalamus representing the
eye goes to two v1 and it onlygoes a very small part of v1.
It connects to a few columns andthat's it.
But these, a diffuse one wouldgo to like all of v1.
It just, it goes up in thecortex and it just spreads like
a big, you know, bearing tree orsomething like that.
Matt (07:21):
It distributes it across the whole thing
Jeff (07:21):
Crawls all it.
So it's not, it can't be sendingsomething very specific because
it goes everywhere.
Now the, the, the clever thingabout these relays, these matrix
cells, they, um, they, they looklike, uh, you know, they'll,
they'll be a bunch of them andthat bunch will project to like
all of these individual regionsand there's another bunch that
projects to all the differentauditory regions.
So that's a second type of cell.
(07:44):
And then there's a third thingthat goes on in the thalamus,
which is the thalamus sets upthese rhythms or cycles, like a
frequency between the cortex andthe thalamus called gamma
frequency.
So what the hell is going onhere?
Matt (07:59):
This is an oscillation, right?
Jeff (08:02):
It's oscillation I'm sorry, I should use that word:
oscillation.
So, um, we've been wonderingabout all these things and also
as we studied the Neocortex, wehave certain functional things
we need to get done.
We know, oh, we need to learnsequences, we have to make
predictions, we have to do motorbehavior and so on, whatever.
And we know that some of that'sgoing to be going on in the
thalamus.
So we have like, but we don'tknow which part.
(08:23):
So we said, okay, we've gotthese functional and
requirements, we're trying tomatch them to the neuroscience.
We've got this crazy thingcalled the thalamus, which has,
you know, unknown properties,unknown functions, and, but we
know it's going to be important.
And by the way, you know, ifyou, if you don't have a
thalamus, your vegetablebasically, literally you're just
a vegetable.
So it's an essential organ.
Yeah.
Um, and um, and so we've beenstruggling for a long time to
(08:46):
figure out what to do with thesethings.
Now, I had a theory for one ofthese components a long time ago
and I still stick to it.
The matrix cells.
And if, if you're long timeNumenta follower you might know
that I brought this up in bothon the forum and I've talked
about in various places, um,I've been talking about this for
many years now, that I had, whenwe came up with the sequence
(09:08):
memory algorithm or even beforethat and before we figured it
out, one of the things we knew,that sequences in the brain have
to be able to do is you have tobe able to speed them up and
slow them down, right?
Like I could play melody backfaster, I could recognize it
faster.
I could speak faster and slower.
So there's all these
Matt (09:22):
Yeah, and it's not different patterns.
It's the same pattern.
Jeff (09:24):
It's the same time, but you can speed up or slow it
down.
And so, there had to be amechanism for that in the brain
and if you think about it, thatmechanism has to apply not to
one column in the cortex.
It would apply to like all thecolumns that are doing, that are
doing speech, something likethat.
So you would want a broad,timing signal and, and so the
(09:48):
system would have to have someway of sort of setting a time or
a time rate for a large area ofthe cortex.
And there aren't too many placesI could do this, right?
So the matrix cells, um, uh,were very suggestive, their
anatomy.
Matt (10:02):
Because they're diffuse.
Jeff (10:03):
They're diffuse and because they're not diffuse
everywhere, it's not like it wasover the whole neocortex.
It'd be over, like auditory,visual or tactile would be
separate sets of matrix cells.
There are cells that go over theentire cortex that might be for
really seeing a neuro modulators
Matt (10:19):
From the Thalamus?
Jeff (10:20):
No, from other places.
You know, like if you werefeeling certain emotions, well
that can be everywhere, right?
But, but this was very specificit, but it's broad, but it's
also specific to modality andalso what happens, is it all
parts of the neocortex projectto the thalamus.
These are not, they will projectto the Thalamus and then there's
thalamus projects back.
So it's as if all the parts ofthe cortex could say, I need to
(10:44):
tell you how fast to go or slowto go and everyone, we all have
to go at the same speed.
So I have speculated numerousover, over probably 15 years or
20 years that perhaps thesematrix cells are involved in
timing and, and every time I goto a conference and I speak to
someone who knows somethingabout the thalamus and the
matrix cells and not that manypeople who do this, I run this
(11:06):
idea by them.
And so far I've been encouragedthat this is the people have
said, yeah, that's a reasonablehypothesis.
Or, here's a piece of data whichsupports it.
Uh, for example, on certainanimals, every time they start a
movement, there's a pulse ofactivity in these matrix cells.
So that would be like starting,the way I think it works this
timing works, is that you, youstart like a clock every time
(11:28):
you have a note in the melodyand you can say, okay, from this
note, how long do I have to waittill the next one?
And then I started the next one.
Now the next one and things likethat.
So there's some indicationsthat's right.
Um, I think you're going to talkto Subutai later about some
theories we have about the relaycells.
Matt (11:42):
I will.
And that'll probably be the nextpodcast after this one, so I
will talk to him about that.
Jeff (11:47):
We have, we are now developing a theory of what the
relay cells are doing, so nowwe've had two set of theories.
One is like what the relay cellsare doing as sort of a remapping
of these, of these relay cells.
And we have a theory about whatthe matrix cells are doing,
which is timing.
These are speculative, but we,there's some reasons to believe
that might be true.
(12:08):
And now I'm going to talk to youabout a third one.
And this has to do again withsort of timing, but it has to do
with those oscillations.
Matt (12:16):
Okay.
Okay.
The gamma frequencyoscillations.
Jeff (12:19):
Yeah.
Right.
Now we're going to dig a littlebit here.
If you've been following ourwork, you know that we think
there are grid cells or gridcell like cells in the
neocortex, everywhere.
And they are, this is like ahuge discovery I think.
And it's like to explains howthe cortex models the world, and
basically creates referenceframes for everything.
And grid cells of course existin an old part of the brain.
(12:42):
That's where they were firstdiscovered in the, in the
entorhinal cortex.
And that's for learning maps ofthe world.
Like when you're walking aroundlike where am I in this office?
But then we think the grid cellsin the cortex are being used to
map things like my coffee cupand you know, and spaces around
my body and things like that.
So that's a big theory.
We published papers on this andum, now we propose that there
are these cells throughouteverywhere in the neocortex,
(13:03):
every cortical column there aregrid cells, cortical grid cells.
How do grid cells work?
Well, that's still a little bitof a mystery.
We know a lot.
We as in the broader generalneuroscience community knows a
lot about uh, how they behaveand how they make maps of the
world and so on.
There's a tremendous amount ofliterature on, on grid cells,
but the exact details of howthey do what they do is still
(13:25):
uncertain.
And one of the leading theoriesis that grid cells work on
oscillations.
So, um, if I want to move in theworld and the grid cells are
representing my position So whatthat means is a bunch of cells
and some of them active.
And as long as I don't movethose cell, say active for
minutes, they're saying likeright now I can close my eyes
and say, Oh, I know in my officeI know where am I and where I am
(13:47):
in my office.
And if I slide my, you know, mychair to the right, now I'm, I'm
in a different place, right?
So those are grid cells.
And um, how do they know tochange their activity when I
move?
Well one of the leading theoriesis that there's an oscillation
and the grid cells, and um, ifyou speed the oscillation up,
it's like movement.
(14:07):
It moves, it moves a bump ofactivity of cells.
So there's an oscillation goingon.
And when you change, there's twoactually oscillations.
And when you change the relativefrequency, it is equivalent to
moving.
So when you move, this theorysays that when you move, how do
the grid cells change theactivity in this very clever way
they do it?
The way they do it is they havethese two oscillations and they
(14:30):
speed one up and slow it down.
It's like a relative frequency.
It's like it's, if you knowmusic, it's like your two notes,
two notes are in tune.
And then one gets a little flat,one gets a little sharp and you
hear this beating frequency.
Matt (14:42):
Right.
But that just gives you an up ora down.
Jeff (14:45):
Yeah.
But it's more complicated thanthat.
There's also an orientation andit tells you which way you're
going.
But let's not try to understandthis theory about grid cells
because it's really complicatedand it's, it's hard to describe.
But anyway, there's a general,there's a lot of clues that say
grid cells are dependent onoscillating frequencies, and
(15:06):
that the change in thosefrequencies indicates you're
moving.
The change in two frequencies,indicates you're moving, and it
can tell you which directionyou're moving.
So this is an ongoing area ofresearch in the entorhinal
cortex.
All right?
Now we're going back now to gridcells in the neocortex.
Okay.
Um, and what if they work on thesame principles?
(15:26):
They work on the sameprinciples, the oscillations.
Well, there's an oscillationfrequency set up between the
thalamus and, um, then theneocortex, it's what they call
gamma frequencies and no oneknows what they're for.
They seem to be related toattention.
So when you attend to somethinglike, Oh, I'm not just looking
at you and now I'm looking atyour nose, or I'm looking at
(15:47):
your ear, you know, zooming inon something.
And the thalamus has long beenimplicated and attention and the
other, and then what you'regoing to talk to Subutai about
is also about attention.
Matt (15:55):
It seems to be in a good location.
Jeff (15:57):
It is a good location.
It's sorta like if there wassomebody in control of like, you
know, where are you going tolook now?
it would be the thalamus.
Okay.
So, um, so, uh, one of thethings we, I was, I was going
through some of our corticaltheories recently and, well, you
know, we talked about like,imagine we talked about the
coffee cup and we talked abouttouching the coffee cup.
Yes.
And uh, that's what we describedin our paper last December.
(16:20):
And now what if I gave you acoffee cup that was smaller?
A lot smaller half the size.
It's just a little child'sCoffee Cup, tea set.
Matt (16:33):
Shot Glass.
Jeff (16:34):
Well, let's make it a coffee cup still.
Okay.
So I want it to be the same.
Maybe shot glass size.
Matt (16:40):
Shot glass size.
Jeff (16:40):
Okay.
Yes.
We don't want little kidsdrinking shots.
But imagine it's the same CoffeeCup, but it's just shrunk down.
Honey, I shrunk the Coffee Cupand you'd look at that.
You'd see it's a Numenta CoffeeCup and you could, you could
pick it up and use it.
Matt (16:55):
I could
Jeff (16:56):
Right, and you could make predictions about it.
Now, that's a different object,but you see it as the same
object, but now think about itas you manipulate that object as
you, as you grab the handle ormove the finger to the other
side, you don't have to moveyour fingers much.
Right?
If I want to make- everything islike shrunk down, all my
movements would be shrunk down.
Even when I'm looking at it, ifI'm moving my eyes to look for
(17:18):
different parts of the CoffeeCup, I move my eyes less because
they're going- it's half as big.
I have to move half as much.
If I want to move my finger fromthe lip of the Coffee Cup to the
handle, I have to move it halfas much.
Yet, I don't have a problemdoing that.
It's like I have a model of theCoffee Cup and I bottle it says,
Hey, I know what this Coffee Cupis, but today I've shrunken the
Coffee Cup and everything has tobe scaled.
Matt (17:40):
Yeah.
Yeah.
Jeff (17:41):
Okay.
How do you do that?
I mean the way we think aboutgrid cells, we don't really
think that's possible.
So, um, so the idea is thefollowing.
This is the idea I had.
I said what if part of theattention, part of what the
thalamus does is establishesthese frequencies between the
cortex and the thalamus, andwhat if it could change those
(18:01):
frequencies based on scale?
Matt (18:03):
Right.
Jeff (18:05):
So it's scaling movements.
It's scaling space.
Matt (18:08):
So it's scaling objects, too.
Jeff (18:10):
Yes.
So, so yes.
So this is, so the idea I hadbefore is that thalamus matrix
cells would be scaling time andin this case they'd be scaling
sort of space.
Like, like how far, if I move myfinger as I typically do from
the Coffee Cup handle to the lid- Today, I want to move it, but
(18:31):
I only want it to go half as farbecause everything is shrunk.
Matt (18:34):
That's a lightbulb.
That makes sense to me becausemovement is space.
That's how we learn space isthrough movement.
Jeff (18:41):
Yes.
Well the whole, exactly.
So everything, the whole theorythat we've got here is that the
Cortex is this, you know,basically modeling space using
grid cells and place cells andso on, and you learn movements
through those spaces.
Matt (18:53):
Exactly.
Jeff (18:53):
Um, and in fact, um, um, I just wrote the chapter in my new
book about this when I wastalking about how moving through
spaces, that's what you do whenyou think.
You know, when you're actuallymoving through a space.
Yeah.
I think you and I may havetalked about that once.
Matt (19:09):
We have, yeah.
It's a great- Absolutely.
That's the way I think.
Jeff (19:13):
So anyway, so now this, this is the idea.
So now that's as far as I'vegone with the idea.
I have a couple of paperssitting in front of me that, um,
that we got from a colleague atMIT who studies the thalamus
about gamma frequencies.
Matt (19:27):
Do you want to mention them?
Jeff (19:28):
No, I haven't even read them yet.
I don't want to go there yet.
This is very speculative, thiswhole thing.
You know I wouldn't be surprisedif six months I said, you know
Matt, that was a great idea, butit was wrong.
Yeah.
So, um, but the more I've beenthinking about it, the more I
think it's probably right or insome sense right that uh, we,
there was a functional need forscaling.
(19:49):
I've sort of identified that.
Somehow our models of the worldhave to be able to, you'd have
to build a scale them underdifferent situation.
You have to be able to deformthem.
Part of that is, is the movementyou use to interact with those
things.
And so you have to scale yourmovements and the size of things
and the movements are all basedon grid cells and grid cells are
(20:11):
based on oscillations.
And so if you can change theoscillating frequencies, you
would basically in one fellswoop change the scale of
everything.
And now you have this thing inthe middle of the brain, the
thalamus, which makes theseprojections, sets up these
oscillations between differentparts of the- broadly.
These were not, these wereagain, broad oscillations
(20:32):
between visual system, betweenthe auditory system and so on.
And therefore you have a, asystem, a set of neural tissue,
the thalamus, which is in agreat position to change
oscillations, which would changethe scale of grid cells
throughout that whole area.
So now that I can just veryquickly scale my thinking about
any particular object in thesame way I could scale my time
in the melody.
(20:53):
So, so it could be like, oh,right, what's the thalamus
doing?
Well the thalamus might be doingthree things that we've
identified so far.
One is scaling time, speedthings up and slow them down at
time.
Like a melody or like my speech,I could talk slower.
It might be easier if I talkslower.
I'm saying the same words.
I can talk really fast, too andI could scale, um, uh, the, um,
(21:18):
my movement.
I'll give you another example ofscaling movements.
Like if you write, uh, with apen.
Here, I see you're writingsomething with a pen and, and
you're signing your name orsomething like that.
Well, um, I can sign my namebigger.
I can just make more grandiosemovements and I get the same
signature.
Matt (21:36):
Yeah, yeah.
Jeff (21:36):
Um, it's like the same motor command being played back,
but I'm just paying it back withmore movement, more scale.
So we have these three thingsgoing on.
We think, we think that thematrix cells might be a modifier
basically scaling time.
Uh, we have these oscillationsthat are established between the
thalamus and the neocortex couldbe scaling space and movements
(21:57):
in those spaces.
It's really the same thing.
And then, um, Subutai's going totalk about later that the relays
tells we think are a way of sortof routing information and uh,
it might also be related toscaling.
Um, but uh, I don't know whathe's going to say about that.
Matt (22:15):
This is fascinating because it really connects time
and space.
Like right there in thethalamus.
It connects time and space.
It's just fascinating.
Jeff (22:25):
I know you like time and space.
Matt (22:26):
Through movement.
Jeff (22:26):
I know you love time and space.
Matt (22:27):
I love time and space!
Jeff (22:28):
I know you do, Matt.
And it's great.
I do too.
But I know you're reallyfascinated, but it's, I never
thought about that as like,yeah, it's a, you know, you can
think of the thalamus as thespace time coordinator, you
know, it's like, or space,space, time warping system.
Matt (22:41):
Well, there's a lot.
Yeah, exactly.
Jeff (22:44):
It's a time warp.
Matt (22:45):
I've read a lot of people think that if we're
experiencing, you know, Ialways, I hate to use this term,
but a qualia, a sensation, likethe sense of now being present,
that it's probably in thethalamus where that sort of
originates.
Jeff (22:57):
Well, I wouldn't go there, but you might want to go to
there.
I think.
I think equalia has a differentmeaning than you just said that,
but it's a very debated term.
So we can leave that for afuture.
Matt (23:06):
Let's leave that for a future podcast.
That sounds good.
Jeff (23:08):
I don't know if it's true about that.
I mean, I think that the risk ofsomething like that we all, we,
people tend to want to thinkabout, oh, where am I in the
brain.
Right.
And, um, and that's a veryslippery slope because then you
end up thinking like, well,there's a little, you know, the
old humunculus and there's somelittle person that I'm looking
at.
Matt (23:26):
I never go there.
Jeff (23:28):
I know but if you're trying to isolate where you
are...
Matt (23:30):
You have to realize without the neocortex, the
thalamus wouldn't work.
Right.
I mean, so you can't say thatjust what you need.
Jeff (23:36):
I just put it this way.
I don't think, I'm not surethere's a location for those
things you're thinking about.
But it is true that the, and theway I think about it now, the
thalamus, I actually think thethalamus is part of the
neocortex.
That's how I think about it.
I mean physically it's not.
Physically, it's connectedmassively, but it's not
physically the same tissue.
Matt (23:54):
We have to understand it to understand the cortical
circuit, right?
Jeff (23:56):
Yeah.
I just, I, it's, it's almost asif, um, there was another layer
of cells in the neocortex, butnormally that would be spread
over this big area.
And we wanted this layer cellsto be all brought together in
one spot so that we can, we canhave, we can act on that layer
all at once.
Where if it was on the bottom ofthe necortex, there's no,
(24:16):
there's no connections that goall over the place, but bringing
all those cells into one spot,now I can say, okay, together
we're going to speed up.
Together, we're going to stretchtime.
Together we're gonna stretch-
Matt (24:26):
Space.
Same thing.
Jeff (24:26):
Yeah.
And so that's just a metaphorfor the way to think about it.
Matt (24:32):
So like time warping is similar to space warping in that
aspect.
Jeff (24:35):
It's almost, yeah.
You know, and of course we knowtime and space are the same
really, some might say so.
So yeah.
Um, but here I think they, ifthese theories are correct and
there are separate cells that dothe time warping and the space
warping and, and um, and thenthere's the relay cells.
So that's, um, I think, youknow, one of my, I think I
almost 40 years I've beenbothered by the thalamus because
(24:59):
it's one of the first things I,you know, if you start studying
the anatomy of the brain, yourealize that it's an, it's a
structure you can't ignore.
And it's been a mystery, uh,what it does, and there have
been numerous teams over theyears who've written about the
thalamus- huge.
I have some monstrous booksabout the pharmacists,
thousands, probably thousands ofpapers, at least hundreds of
papers about the thalamus, andyet almost nothing about what it
(25:22):
actually does and how it mightfunction.
And so there's a lot ofspeculative theories that
suggest it's related toattentional mechanisms, but you
know, it was very little data onthat.
So anyway, I feel excited thatthis is all of a sudden as you,
just as you were expressing itfor this very recently in the
last few months.
Um, I mean that it's maybe allcoming together.
(25:42):
Yeah.
I've known about the matrixsales for a long time, but, uh,
what the relay cells are doing,which Subutai'll talk to you
about and what the oscillationsmight be doing.
These are all speculative, butnow all of a sudden there's a
sort of a cohesive theory about,yeah, it is a tension, but it's
also scaling and it's also, um,uh, scaling space and time and
that was necessary and it allowsus to take what we know about
(26:05):
the world and shrink it andexpand and kinda speed up and
slow it down to fit the currentsituation.
Matt (26:12):
Yeah.
It's functions we have to bedoing somehow.
Jeff (26:14):
Yeah.
Yeah.
It's just as simple as justlistening to someone speak, you
know, there's a model of what aword is and some people speak
faster and some people speakslower and that model has to be
stretched.
And so once I started listeningto you at a certain rate, then I
might be working at that ratefor a while and maybe speed up
and quickly adjust.
You can think of just like amelody, right?
(26:35):
If you are listening to amelody, um, you learned it at
one tempo and now you're hearingit in a new tempo, very quickly
you can catch onto it.
Um, but, uh, but if you, if you,if every note has its own tempo,
then you've lost the melodybecause then basically the
rhythm is lost.
Matt (26:51):
You can't find the intervals.
Jeff (26:52):
Yeah, that's right.
So, so you kind of have to, ithas to last for awhile, you
know, and then, but you're goingto get, as you're going along,
if someone wants us to, youknow, change the tempo in the
middle of the song they can dothat.
And you go, oh, it's like asurprise.
Oh, look at that.
Matt (27:06):
It can be pleasant.
Jeff (27:06):
It is pleasant.
And it's the same as changingthe key.
Key shifts.
You can do a temporal shift inthe middle of the song.
What's that song?
A little bit slower now?
A little bit slower now.
So, yeah.
So that's basically, um, uh, thetopic that I was going to talk
about today.
Matt (27:21):
Well, you mentioned a book you're writing.
Did you want to say somethingelse about that?
Jeff (27:24):
Uh, I'm just writing another book.
Um, and I want to say too muchabout it at this point in time.
We could do a whole talk.
We can do a whole podcast aboutthat.
Matt (27:32):
Maybe later then.
Jeff (27:33):
Uh, yeah, it, it's basically a, it's, it's, it's
sort of a follow-on to Onintelligence, but with all the
stuff we've learned now abouthow the cortex works, so it's On
Intelligence was more of a callto like, we should solve how the
brain works, we should solve howthe neocortex works.
It's going to be important forAI, it's gonna be important for
a lot of things.
And here's some ideas about howit might work.
Now it's like, oh, we figuredout a lot of it.
(27:56):
And, um, and let me tell youwhat it is.
And let me sort of go into depthabout it.
And that's half the book.
And the other half of the bookis implications of, um, what
we've learned, the implicationsfor society.
Matt (28:06):
Well, I know a lot of people are looking forward to
it, so I'm excited that you'redoing it.
Jeff (28:10):
Yeah.
Well, it just doesn't remind youthat writing a book takes a long
time.
So it may be, it easily could bea year before this book
surfaces, even though I'mworking on it everyday.
Matt (28:20):
Right.
Well, hopefully we can do moreof these chats as we go on.
Jeff (28:24):
Yeah.
If this works, we'll find out ifit works.
Matt (28:26):
We'll find out.
Jeff (28:27):
Um, there's lots of speculative topics like this.
This is what we do here everyday.
Matt (28:31):
All right, well if you want to hear more about it, then
make sure and contact us.
Leave a comment on our podcastor email me at matt@numenta.org
even.
I'd be happy to hear yourfeedback.
Jeff (28:41):
All right.
Matt (28:41):
Thanks, Jeff.
Jeff (28:41):
That was fun.
Matt (28:42):
This has been a pleasure.
Yeah.
One of the things we knew is that sequences in the brain
have to be able to do is youhave to be able to speed them
up, slow them down.
Matt (00:04):
Right.
Yeah.
Jeff (00:05):
Like I can play a melody back faster.
I can recognize it faster.
I can speak faster and slower.
So there's all these-
Matt (00:09):
And it's not different patterns.
It's the same pattern.
Jeff (00:11):
Yeah.
It's the same pattern.
But you can speed up and slow itdown and so, um, I, there had to
be a mechanism for that in thebrain and if you think about it,
that mechanism has to apply notto one column in the cortex, it
would apply to like all thecolumns that are doing, that are
doing speech, something likethat.
Christy (00:35):
You’re listening to Numenta On Intelligence, our
podcast series on howintelligence works in the brain
and how to implement it innon-biological systems.
In this episode, we’re going totake you back to where the
podcast started– a conversationwith our co-founder Jeff
Hawkins.
In fact, for the next fewepisodes, we’re going to try
something a little differentthan the previous few episodes,
which featured Interviews withNeuroscientists.
Matt Taylor, our Open SourceCommunity manager, will be
bringing you conversations withJeff Hawkins, or Subutai Ahmad,
our VP of Research, on a varietyof topics.
These could include anythingfrom Jeff’s latest research
developments that haven’t yetbeen documented, to Subutai’s
updates on applying our theoriesto today’s machine learning
systems.
These will be casualconversations, the kind that
happen every day here atNumenta, and we want to share
them with you, our listeners.
Some of them may get prettytechnical, but we’ll provide
links to resources for furtherreading.
I hope you enjoy theseconversation from Numenta On
Intelligence.
Matt (01:34):
So I'm here with Jeff Hawkins in his office.
It's early in the morning hereat Numenta.
There's just me and you andwe're going to talk about some
speculative thoughts andthinking you have about the
thalamus, right?
Jeff (01:45):
Yeah.
Yeah.
So, uh, we chosen, I've chosen,uh, to talk about the Thalamus
now.
I'll tell you what it is and whywe think it's important.
And then some new ideas we haveabout what it might be doing.
Matt (01:57):
I think people will be interested cause we, we voted,
we usually just talk aboutneocortex.
So it's a little refreshing togo to somewhere else in the
brain.
Jeff (02:04):
Yeah.
So let's just paint a picture inyour mind.
You know, the neocortex is, islike a big sheet, like a big
napkin wrinkled up on top ofyour brain.
And it's only a couple ofmillimeters thick.
The thalamus, it's right in thecenter of the brain and it's two
parts, like everything in thebrain.
And there's like maybe like theshape of two eggs, two small
eggs.
Matt (02:22):
Right.
One on either end, eitherhemisphere.
Right?
Jeff (02:24):
So we, you know, that's everything in the brain has got
two parts.
We don't usually focus on that,but they're doing the same
thing, just divided in half.
So the thalamus is right in themiddle and um, it has a very,
uh, unique relationship to theneocortex.
In fact, myself and some otherpeople studying the thalamus
really don't think you couldseparate them.
Um, and the reason is, is thatwhen information goes to the
(02:46):
neocortex, from your eyes, yourskin or your, or your ears, it
always goes through thethalamus, always.
Matt (02:53):
Sensory information...
Jeff (02:53):
All sensory information, pretty much, pretty much almost
any sensory information.
And so the object nerve, itdoesn't go from the back of the
retina, it doesn't go straightto the cortex.
It goes to the thalamus and thethalamus then goes to the
cortex.
Matt (03:05):
Right.
Jeff (03:06):
And when two regions in the neocortex project to each
other, you know, region A andregion B or v1, v2, there's they
primarily do that through thethalamus again.
So v1 projects to the thalamusand then the thalamus predicts
to v2 that there are directconnections between these
cortical regions, but there'salways one that goes through the
thalamus.
And since all information goesthrough the thalamus, it used to
(03:29):
be thought that the thalamus islike the gateway to the cortex.
Like when you get through thisgateway, you're in.
But now it's believed thateverything that goes between
cortical region, the corticalregion goes through the
Thalamus.
And that way you could think ofit as a part of the neocortex.
I kind of think of it like anextra layer of cells or extra
set of tissue that has been,that's been consolidated down
(03:53):
the center of the brain.
But it's almost, it's sointimately connected with the
neocortex.
You can't really separate themout.
Matt (03:58):
But it is an older part of the brain, right?
Jeff (04:01):
Ah, I don't know that's true actually, Matt.
I, you know, it's in the sensethat it's not neocortex.
We often tend to think abouteverything that's not the
neocortex is the older part ofthe brain.
I don't know if the thalamusexisted prior to the neocortex.
I don't know that.
Yeah.
So, um, it probably did in someform, but really I think we, if
you really want to understandhow the neocortex works,
(04:22):
ultimately you have tounderstand how the, what the
thalamus is doing.
Matt (04:25):
There are no mammals that we can find that just have a
thalamus and no neocortex
Jeff (04:29):
No mammals, all mammals have a neocortex of the question
is, or the non mammals that havea thalamus that don't have an
neocortex.
I don't know the answer thatone, but we can, we can pretty
much say right now the neocortexis intimate and required, and
the thalamus, is required tohave a neocortex.
So you can't really separate thetwo out.
Matt (04:48):
Well definitely the projections show that
Jeff (04:50):
That's right.
The projections show that,physically it's separated.
But the projections are veryintimate and um, and so we've
always known that any theory ofneocortex is going to have to
explain what the thalamus isdoing.
Um, and the thalamus itself isnot super complicated, but it's
not super simple either.
And I'm going to break it intotwo broad categories.
(05:10):
There's one broad category we'vejust been talking about, which
are called relay cells
Matt (05:16):
Relay cells.
Jeff (05:17):
Yeah, so like an axon comes from your eyeball and
Matt (05:20):
Oh it's synapses in the thalamus...
Jeff (05:22):
It makes a connection to one of these relay cells in the
thalamus and the relay cells inthe thalamus goes to the
neocortex and it literally lookslike one to one.
Matt (05:30):
Wow.
So just one, one stop and then
Jeff (05:32):
That's right and not only one stop it.
It's, there's very little con-,there's no convergence in the
thalamus.
It's like it's literally likeyou took a long wire and you cut
it and cut it in half and thenconnect it back together.
Matt (05:44):
So why do it?
Jeff (05:44):
That's right.
Right.
That's a good question.
So the call of the relay is alittle bit misleading because we
know it must be, there's noreason you're not going to have
this to do nothing.
Right.
But that's kind of what it lookslike.
They can say, this will show inmany situations, it's one to one
correspondence.
A spike comes in and spike comesout and there's a, the topology
of the arrangement of the cellsin the retina are preserved in
(06:04):
the Thalamus and they project tothe cortex and they're preserved
in the cortex.
So, one thing is why are thererelay cells?
The second thing, there's awhole bunch of other cells and
by the way it was a relay cells,a group of relay cells that go
between the retina and V1-that's called LGN lateral
Janeka.
No, cause that's just a bunch ofcells in the Thalamus and the
separate ones that go betweenall the other regions.
(06:25):
So these are dedicated cells forthese units.
Matt (06:26):
Right.
They're localized to
Jeff (06:29):
to whatever that projection is.
There's another set of cells inthe thalamus which are much more
diffuse.
They project to the cortex andthey project very diffusely.
So these cells, uh, they go bydifferent names.
I prefer the one that sometimesthey're called matrix cells.
I'm not referring to the movie,but referring to the, uh, the
fact that they're sort ofinterstitial to these other,
(06:51):
they're sort of in the matrix ofthe thalamus.
Matt (06:53):
When you say diffuse, just to define that term and the
connection is diffuse that'slike a direct...
Jeff (06:58):
Yeah.
So let's say a relay cell fromthe thalamus representing the
eye goes to two v1 and it onlygoes a very small part of v1.
It connects to a few columns andthat's it.
But these, a diffuse one wouldgo to like all of v1.
It just, it goes up in thecortex and it just spreads like
a big, you know, bearing tree orsomething like that.
Matt (07:21):
It distributes it across the whole thing
Jeff (07:21):
Crawls all it.
So it's not, it can't be sendingsomething very specific because
it goes everywhere.
Now the, the, the clever thingabout these relays, these matrix
cells, they, um, they, they looklike, uh, you know, they'll,
they'll be a bunch of them andthat bunch will project to like
all of these individual regionsand there's another bunch that
projects to all the differentauditory regions.
So that's a second type of cell.
(07:44):
And then there's a third thingthat goes on in the thalamus,
which is the thalamus sets upthese rhythms or cycles, like a
frequency between the cortex andthe thalamus called gamma
frequency.
So what the hell is going onhere?
Matt (07:59):
This is an oscillation, right?
Jeff (08:02):
It's oscillation I'm sorry, I should use that word:
oscillation.
So, um, we've been wonderingabout all these things and also
as we studied the Neocortex, wehave certain functional things
we need to get done.
We know, oh, we need to learnsequences, we have to make
predictions, we have to do motorbehavior and so on, whatever.
And we know that some of that'sgoing to be going on in the
thalamus.
So we have like, but we don'tknow which part.
(08:23):
So we said, okay, we've gotthese functional and
requirements, we're trying tomatch them to the neuroscience.
We've got this crazy thingcalled the thalamus, which has,
you know, unknown properties,unknown functions, and, but we
know it's going to be important.
And by the way, you know, ifyou, if you don't have a
thalamus, your vegetablebasically, literally you're just
a vegetable.
So it's an essential organ.
Yeah.
Um, and um, and so we've beenstruggling for a long time to
(08:46):
figure out what to do with thesethings.
Now, I had a theory for one ofthese components a long time ago
and I still stick to it.
The matrix cells.
And if, if you're long timeNumenta follower you might know
that I brought this up in bothon the forum and I've talked
about in various places, um,I've been talking about this for
many years now, that I had, whenwe came up with the sequence
(09:08):
memory algorithm or even beforethat and before we figured it
out, one of the things we knew,that sequences in the brain have
to be able to do is you have tobe able to speed them up and
slow them down, right?
Like I could play melody backfaster, I could recognize it
faster.
I could speak faster and slower.
So there's all these
Matt (09:22):
Yeah, and it's not different patterns.
It's the same pattern.
Jeff (09:24):
It's the same time, but you can speed up or slow it
down.
And so, there had to be amechanism for that in the brain
and if you think about it, thatmechanism has to apply not to
one column in the cortex.
It would apply to like all thecolumns that are doing, that are
doing speech, something likethat.
So you would want a broad,timing signal and, and so the
(09:48):
system would have to have someway of sort of setting a time or
a time rate for a large area ofthe cortex.
And there aren't too many placesI could do this, right?
So the matrix cells, um, uh,were very suggestive, their
anatomy.
Matt (10:02):
Because they're diffuse.
Jeff (10:03):
They're diffuse and because they're not diffuse
everywhere, it's not like it wasover the whole neocortex.
It'd be over, like auditory,visual or tactile would be
separate sets of matrix cells.
There are cells that go over theentire cortex that might be for
really seeing a neuro modulators
Matt (10:19):
From the Thalamus?
Jeff (10:20):
No, from other places.
You know, like if you werefeeling certain emotions, well
that can be everywhere, right?
But, but this was very specificit, but it's broad, but it's
also specific to modality andalso what happens, is it all
parts of the neocortex projectto the thalamus.
These are not, they will projectto the Thalamus and then there's
thalamus projects back.
So it's as if all the parts ofthe cortex could say, I need to
(10:44):
tell you how fast to go or slowto go and everyone, we all have
to go at the same speed.
So I have speculated numerousover, over probably 15 years or
20 years that perhaps thesematrix cells are involved in
timing and, and every time I goto a conference and I speak to
someone who knows somethingabout the thalamus and the
matrix cells and not that manypeople who do this, I run this
(11:06):
idea by them.
And so far I've been encouragedthat this is the people have
said, yeah, that's a reasonablehypothesis.
Or, here's a piece of data whichsupports it.
Uh, for example, on certainanimals, every time they start a
movement, there's a pulse ofactivity in these matrix cells.
So that would be like starting,the way I think it works this
timing works, is that you, youstart like a clock every time
(11:28):
you have a note in the melodyand you can say, okay, from this
note, how long do I have to waittill the next one?
And then I started the next one.
Now the next one and things likethat.
So there's some indicationsthat's right.
Um, I think you're going to talkto Subutai later about some
theories we have about the relaycells.
Matt (11:42):
I will.
And that'll probably be the nextpodcast after this one, so I
will talk to him about that.
Jeff (11:47):
We have, we are now developing a theory of what the
relay cells are doing, so nowwe've had two set of theories.
One is like what the relay cellsare doing as sort of a remapping
of these, of these relay cells.
And we have a theory about whatthe matrix cells are doing,
which is timing.
These are speculative, but we,there's some reasons to believe
that might be true.
(12:08):
And now I'm going to talk to youabout a third one.
And this has to do again withsort of timing, but it has to do
with those oscillations.
Matt (12:16):
Okay.
Okay.
The gamma frequencyoscillations.
Jeff (12:19):
Yeah.
Right.
Now we're going to dig a littlebit here.
If you've been following ourwork, you know that we think
there are grid cells or gridcell like cells in the
neocortex, everywhere.
And they are, this is like ahuge discovery I think.
And it's like to explains howthe cortex models the world, and
basically creates referenceframes for everything.
And grid cells of course existin an old part of the brain.
(12:42):
That's where they were firstdiscovered in the, in the
entorhinal cortex.
And that's for learning maps ofthe world.
Like when you're walking aroundlike where am I in this office?
But then we think the grid cellsin the cortex are being used to
map things like my coffee cupand you know, and spaces around
my body and things like that.
So that's a big theory.
We published papers on this andum, now we propose that there
are these cells throughouteverywhere in the neocortex,
(13:03):
every cortical column there aregrid cells, cortical grid cells.
How do grid cells work?
Well, that's still a little bitof a mystery.
We know a lot.
We as in the broader generalneuroscience community knows a
lot about uh, how they behaveand how they make maps of the
world and so on.
There's a tremendous amount ofliterature on, on grid cells,
but the exact details of howthey do what they do is still
(13:25):
uncertain.
And one of the leading theoriesis that grid cells work on
oscillations.
So, um, if I want to move in theworld and the grid cells are
representing my position So whatthat means is a bunch of cells
and some of them active.
And as long as I don't movethose cell, say active for
minutes, they're saying likeright now I can close my eyes
and say, Oh, I know in my officeI know where am I and where I am
(13:47):
in my office.
And if I slide my, you know, mychair to the right, now I'm, I'm
in a different place, right?
So those are grid cells.
And um, how do they know tochange their activity when I
move?
Well one of the leading theoriesis that there's an oscillation
and the grid cells, and um, ifyou speed the oscillation up,
it's like movement.
(14:07):
It moves, it moves a bump ofactivity of cells.
So there's an oscillation goingon.
And when you change, there's twoactually oscillations.
And when you change the relativefrequency, it is equivalent to
moving.
So when you move, this theorysays that when you move, how do
the grid cells change theactivity in this very clever way
they do it?
The way they do it is they havethese two oscillations and they
(14:30):
speed one up and slow it down.
It's like a relative frequency.
It's like it's, if you knowmusic, it's like your two notes,
two notes are in tune.
And then one gets a little flat,one gets a little sharp and you
hear this beating frequency.
Matt (14:42):
Right.
But that just gives you an up ora down.
Jeff (14:45):
Yeah.
But it's more complicated thanthat.
There's also an orientation andit tells you which way you're
going.
But let's not try to understandthis theory about grid cells
because it's really complicatedand it's, it's hard to describe.
But anyway, there's a general,there's a lot of clues that say
grid cells are dependent onoscillating frequencies, and
(15:06):
that the change in thosefrequencies indicates you're
moving.
The change in two frequencies,indicates you're moving, and it
can tell you which directionyou're moving.
So this is an ongoing area ofresearch in the entorhinal
cortex.
All right?
Now we're going back now to gridcells in the neocortex.
Okay.
Um, and what if they work on thesame principles?
(15:26):
They work on the sameprinciples, the oscillations.
Well, there's an oscillationfrequency set up between the
thalamus and, um, then theneocortex, it's what they call
gamma frequencies and no oneknows what they're for.
They seem to be related toattention.
So when you attend to somethinglike, Oh, I'm not just looking
at you and now I'm looking atyour nose, or I'm looking at
(15:47):
your ear, you know, zooming inon something.
And the thalamus has long beenimplicated and attention and the
other, and then what you'regoing to talk to Subutai about
is also about attention.
Matt (15:55):
It seems to be in a good location.
Jeff (15:57):
It is a good location.
It's sorta like if there wassomebody in control of like, you
know, where are you going tolook now?
it would be the thalamus.
Okay.
So, um, so, uh, one of thethings we, I was, I was going
through some of our corticaltheories recently and, well, you
know, we talked about like,imagine we talked about the
coffee cup and we talked abouttouching the coffee cup.
Yes.
And uh, that's what we describedin our paper last December.
(16:20):
And now what if I gave you acoffee cup that was smaller?
A lot smaller half the size.
It's just a little child'sCoffee Cup, tea set.
Matt (16:33):
Shot Glass.
Jeff (16:34):
Well, let's make it a coffee cup still.
Okay.
So I want it to be the same.
Maybe shot glass size.
Matt (16:40):
Shot glass size.
Jeff (16:40):
Okay.
Yes.
We don't want little kidsdrinking shots.
But imagine it's the same CoffeeCup, but it's just shrunk down.
Honey, I shrunk the Coffee Cupand you'd look at that.
You'd see it's a Numenta CoffeeCup and you could, you could
pick it up and use it.
Matt (16:55):
I could
Jeff (16:56):
Right, and you could make predictions about it.
Now, that's a different object,but you see it as the same
object, but now think about itas you manipulate that object as
you, as you grab the handle ormove the finger to the other
side, you don't have to moveyour fingers much.
Right?
If I want to make- everything islike shrunk down, all my
movements would be shrunk down.
Even when I'm looking at it, ifI'm moving my eyes to look for
(17:18):
different parts of the CoffeeCup, I move my eyes less because
they're going- it's half as big.
I have to move half as much.
If I want to move my finger fromthe lip of the Coffee Cup to the
handle, I have to move it halfas much.
Yet, I don't have a problemdoing that.
It's like I have a model of theCoffee Cup and I bottle it says,
Hey, I know what this Coffee Cupis, but today I've shrunken the
Coffee Cup and everything has tobe scaled.
Matt (17:40):
Yeah.
Yeah.
Jeff (17:41):
Okay.
How do you do that?
I mean the way we think aboutgrid cells, we don't really
think that's possible.
So, um, so the idea is thefollowing.
This is the idea I had.
I said what if part of theattention, part of what the
thalamus does is establishesthese frequencies between the
cortex and the thalamus, andwhat if it could change those
(18:01):
frequencies based on scale?
Matt (18:03):
Right.
Jeff (18:05):
So it's scaling movements.
It's scaling space.
Matt (18:08):
So it's scaling objects, too.
Jeff (18:10):
Yes.
So, so yes.
So this is, so the idea I hadbefore is that thalamus matrix
cells would be scaling time andin this case they'd be scaling
sort of space.
Like, like how far, if I move myfinger as I typically do from
the Coffee Cup handle to the lid- Today, I want to move it, but
(18:31):
I only want it to go half as farbecause everything is shrunk.
Matt (18:34):
That's a lightbulb.
That makes sense to me becausemovement is space.
That's how we learn space isthrough movement.
Jeff (18:41):
Yes.
Well the whole, exactly.
So everything, the whole theorythat we've got here is that the
Cortex is this, you know,basically modeling space using
grid cells and place cells andso on, and you learn movements
through those spaces.
Matt (18:53):
Exactly.
Jeff (18:53):
Um, and in fact, um, um, I just wrote the chapter in my new
book about this when I wastalking about how moving through
spaces, that's what you do whenyou think.
You know, when you're actuallymoving through a space.
Yeah.
I think you and I may havetalked about that once.
Matt (19:09):
We have, yeah.
It's a great- Absolutely.
That's the way I think.
Jeff (19:13):
So anyway, so now this, this is the idea.
So now that's as far as I'vegone with the idea.
I have a couple of paperssitting in front of me that, um,
that we got from a colleague atMIT who studies the thalamus
about gamma frequencies.
Matt (19:27):
Do you want to mention them?
Jeff (19:28):
No, I haven't even read them yet.
I don't want to go there yet.
This is very speculative, thiswhole thing.
You know I wouldn't be surprisedif six months I said, you know
Matt, that was a great idea, butit was wrong.
Yeah.
So, um, but the more I've beenthinking about it, the more I
think it's probably right or insome sense right that uh, we,
there was a functional need forscaling.
(19:49):
I've sort of identified that.
Somehow our models of the worldhave to be able to, you'd have
to build a scale them underdifferent situation.
You have to be able to deformthem.
Part of that is, is the movementyou use to interact with those
things.
And so you have to scale yourmovements and the size of things
and the movements are all basedon grid cells and grid cells are
(20:11):
based on oscillations.
And so if you can change theoscillating frequencies, you
would basically in one fellswoop change the scale of
everything.
And now you have this thing inthe middle of the brain, the
thalamus, which makes theseprojections, sets up these
oscillations between differentparts of the- broadly.
These were not, these wereagain, broad oscillations
(20:32):
between visual system, betweenthe auditory system and so on.
And therefore you have a, asystem, a set of neural tissue,
the thalamus, which is in agreat position to change
oscillations, which would changethe scale of grid cells
throughout that whole area.
So now that I can just veryquickly scale my thinking about
any particular object in thesame way I could scale my time
in the melody.
(20:53):
So, so it could be like, oh,right, what's the thalamus
doing?
Well the thalamus might be doingthree things that we've
identified so far.
One is scaling time, speedthings up and slow them down at
time.
Like a melody or like my speech,I could talk slower.
It might be easier if I talkslower.
I'm saying the same words.
I can talk really fast, too andI could scale, um, uh, the, um,
(21:18):
my movement.
I'll give you another example ofscaling movements.
Like if you write, uh, with apen.
Here, I see you're writingsomething with a pen and, and
you're signing your name orsomething like that.
Well, um, I can sign my namebigger.
I can just make more grandiosemovements and I get the same
signature.
Matt (21:36):
Yeah, yeah.
Jeff (21:36):
Um, it's like the same motor command being played back,
but I'm just paying it back withmore movement, more scale.
So we have these three thingsgoing on.
We think, we think that thematrix cells might be a modifier
basically scaling time.
Uh, we have these oscillationsthat are established between the
thalamus and the neocortex couldbe scaling space and movements
(21:57):
in those spaces.
It's really the same thing.
And then, um, Subutai's going totalk about later that the relays
tells we think are a way of sortof routing information and uh,
it might also be related toscaling.
Um, but uh, I don't know whathe's going to say about that.
Matt (22:15):
This is fascinating because it really connects time
and space.
Like right there in thethalamus.
It connects time and space.
It's just fascinating.
Jeff (22:25):
I know you like time and space.
Matt (22:26):
Through movement.
Jeff (22:26):
I know you love time and space.
Matt (22:27):
I love time and space!
Jeff (22:28):
I know you do, Matt.
And it's great.
I do too.
But I know you're reallyfascinated, but it's, I never
thought about that as like,yeah, it's a, you know, you can
think of the thalamus as thespace time coordinator, you
know, it's like, or space,space, time warping system.
Matt (22:41):
Well, there's a lot.
Yeah, exactly.
Jeff (22:44):
It's a time warp.
Matt (22:45):
I've read a lot of people think that if we're
experiencing, you know, Ialways, I hate to use this term,
but a qualia, a sensation, likethe sense of now being present,
that it's probably in thethalamus where that sort of
originates.
Jeff (22:57):
Well, I wouldn't go there, but you might want to go to
there.
I think.
I think equalia has a differentmeaning than you just said that,
but it's a very debated term.
So we can leave that for afuture.
Matt (23:06):
Let's leave that for a future podcast.
That sounds good.
Jeff (23:08):
I don't know if it's true about that.
I mean, I think that the risk ofsomething like that we all, we,
people tend to want to thinkabout, oh, where am I in the
brain.
Right.
And, um, and that's a veryslippery slope because then you
end up thinking like, well,there's a little, you know, the
old humunculus and there's somelittle person that I'm looking
at.
Matt (23:26):
I never go there.
Jeff (23:28):
I know but if you're trying to isolate where you
are...
Matt (23:30):
You have to realize without the neocortex, the
thalamus wouldn't work.
Right.
I mean, so you can't say thatjust what you need.
Jeff (23:36):
I just put it this way.
I don't think, I'm not surethere's a location for those
things you're thinking about.
But it is true that the, and theway I think about it now, the
thalamus, I actually think thethalamus is part of the
neocortex.
That's how I think about it.
I mean physically it's not.
Physically, it's connectedmassively, but it's not
physically the same tissue.
Matt (23:54):
We have to understand it to understand the cortical
circuit, right?
Jeff (23:56):
Yeah.
I just, I, it's, it's almost asif, um, there was another layer
of cells in the neocortex, butnormally that would be spread
over this big area.
And we wanted this layer cellsto be all brought together in
one spot so that we can, we canhave, we can act on that layer
all at once.
Where if it was on the bottom ofthe necortex, there's no,
(24:16):
there's no connections that goall over the place, but bringing
all those cells into one spot,now I can say, okay, together
we're going to speed up.
Together, we're going to stretchtime.
Together we're gonna stretch-
Matt (24:26):
Space.
Same thing.
Jeff (24:26):
Yeah.
And so that's just a metaphorfor the way to think about it.
Matt (24:32):
So like time warping is similar to space warping in that
aspect.
Jeff (24:35):
It's almost, yeah.
You know, and of course we knowtime and space are the same
really, some might say so.
So yeah.
Um, but here I think they, ifthese theories are correct and
there are separate cells that dothe time warping and the space
warping and, and um, and thenthere's the relay cells.
So that's, um, I think, youknow, one of my, I think I
almost 40 years I've beenbothered by the thalamus because
(24:59):
it's one of the first things I,you know, if you start studying
the anatomy of the brain, yourealize that it's an, it's a
structure you can't ignore.
And it's been a mystery, uh,what it does, and there have
been numerous teams over theyears who've written about the
thalamus- huge.
I have some monstrous booksabout the pharmacists,
thousands, probably thousands ofpapers, at least hundreds of
papers about the thalamus, andyet almost nothing about what it
(25:22):
actually does and how it mightfunction.
And so there's a lot ofspeculative theories that
suggest it's related toattentional mechanisms, but you
know, it was very little data onthat.
So anyway, I feel excited thatthis is all of a sudden as you,
just as you were expressing itfor this very recently in the
last few months.
Um, I mean that it's maybe allcoming together.
(25:42):
Yeah.
I've known about the matrixsales for a long time, but, uh,
what the relay cells are doing,which Subutai'll talk to you
about and what the oscillationsmight be doing.
These are all speculative, butnow all of a sudden there's a
sort of a cohesive theory about,yeah, it is a tension, but it's
also scaling and it's also, um,uh, scaling space and time and
that was necessary and it allowsus to take what we know about
(26:05):
the world and shrink it andexpand and kinda speed up and
slow it down to fit the currentsituation.
Matt (26:12):
Yeah.
It's functions we have to bedoing somehow.
Jeff (26:14):
Yeah.
Yeah.
It's just as simple as justlistening to someone speak, you
know, there's a model of what aword is and some people speak
faster and some people speakslower and that model has to be
stretched.
And so once I started listeningto you at a certain rate, then I
might be working at that ratefor a while and maybe speed up
and quickly adjust.
You can think of just like amelody, right?
(26:35):
If you are listening to amelody, um, you learned it at
one tempo and now you're hearingit in a new tempo, very quickly
you can catch onto it.
Um, but, uh, but if you, if you,if every note has its own tempo,
then you've lost the melodybecause then basically the
rhythm is lost.
Matt (26:51):
You can't find the intervals.
Jeff (26:52):
Yeah, that's right.
So, so you kind of have to, ithas to last for awhile, you
know, and then, but you're goingto get, as you're going along,
if someone wants us to, youknow, change the tempo in the
middle of the song they can dothat.
And you go, oh, it's like asurprise.
Oh, look at that.
Matt (27:06):
It can be pleasant.
Jeff (27:06):
It is pleasant.
And it's the same as changingthe key.
Key shifts.
You can do a temporal shift inthe middle of the song.
What's that song?
A little bit slower now?
A little bit slower now.
So, yeah.
So that's basically, um, uh, thetopic that I was going to talk
about today.
Matt (27:21):
Well, you mentioned a book you're writing.
Did you want to say somethingelse about that?
Jeff (27:24):
Uh, I'm just writing another book.
Um, and I want to say too muchabout it at this point in time.
We could do a whole talk.
We can do a whole podcast aboutthat.
Matt (27:32):
Maybe later then.
Jeff (27:33):
Uh, yeah, it, it's basically a, it's, it's, it's
sort of a follow-on to Onintelligence, but with all the
stuff we've learned now abouthow the cortex works, so it's On
Intelligence was more of a callto like, we should solve how the
brain works, we should solve howthe neocortex works.
It's going to be important forAI, it's gonna be important for
a lot of things.
And here's some ideas about howit might work.
Now it's like, oh, we figuredout a lot of it.
(27:56):
And, um, and let me tell youwhat it is.
And let me sort of go into depthabout it.
And that's half the book.
And the other half of the bookis implications of, um, what
we've learned, the implicationsfor society.
Matt (28:06):
Well, I know a lot of people are looking forward to
it, so I'm excited that you'redoing it.
Jeff (28:10):
Yeah.
Well, it just doesn't remind youthat writing a book takes a long
time.
So it may be, it easily could bea year before this book
surfaces, even though I'mworking on it everyday.
Matt (28:20):
Right.
Well, hopefully we can do moreof these chats as we go on.
Jeff (28:24):
Yeah.
If this works, we'll find out ifit works.
Matt (28:26):
We'll find out.
Jeff (28:27):
Um, there's lots of speculative topics like this.
This is what we do here everyday.
Matt (28:31):
All right, well if you want to hear more about it, then
make sure and contact us.
Leave a comment on our podcastor email me at matt@numenta.org
even.
I'd be happy to hear yourfeedback.
Jeff (28:41):
All right.
Matt (28:41):
Thanks, Jeff.
Jeff (28:41):
That was fun.
Matt (28:42):
This has been a pleasure.
Yeah.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Intentionally Disturbing
Join me on this podcast as I navigate the murky waters of human behavior, current events, and personal anecdotes through in-depth interviews with incredible people—all served with a generous helping of sarcasm and satire. After years as a forensic and clinical psychologist, I offer a unique interview style and a low tolerance for bullshit, quickly steering conversations toward depth and darkness. I honor the seriousness while also appreciating wit. I’m your guide through the twisted labyrinth of the human psyche, armed with dark humor and biting wit.