May 9, 2019 • 22 mins
Host Matt Taylor interviews Numenta VP Research Subutai Ahmad about his experience at this year's COSYNE conference. They discuss:
- What COSYNE is all about
- Why Numenta attends
- What Numenta presented
Then Matt dives into the content and asks Subutai about
- the difference between tonic and burst modes
- how cells in the cortex control modes in the thalamus
- why would cortex control modes of thalamus
- how TC cells detect precise sparse codes from L6
- what the thalamus is providing in the FF input to L6
- the hypotheses suggested by the poster
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Matt (00:00):
Today on the Numenta On Intelligence podcast.
Subutai (00:04):
If you're a computer scientist, you might know about
multiplexers where you set aparticular switching function.
Depending on a state, you, youset the system to switch input
in a certain way and when inputcomes in and gets moved around
and switched around, so theBallon boundless could do that
through by control of the viacontrol of the neocortex.
Matt (00:24):
That was Subutai Ahmad on the Thalamus.
I'm Matt Taylor, Numentacommunity manager, and you're
listening to the Numenta OnIntelligence podcast.
I'm here with, Subutai Ahmad ourVP of engineering at Numenta and
he recently attended the COSYNEconference and I wanted to talk
to you about it.
(00:44):
High.
Subutai.
Subutai (00:45):
Hi Matt.
Happy to be here.
Matt (00:47):
Awesome.
So, um, tell me about the COSYNEconference.
What, who attends and what is itall about?
Subutai (00:52):
Yeah, so the COSYNE conference is pretty much the
main conference in computationalsystems, neuroscience.
That's what COSYNE stands for,every year.
And it's probably one of themain conferences that we attend
every year.
Uh, it originally split out fromNIPS, um, way back in the 90s.
I think, um, which is now called[inaudible], it's called Neurips
(01:12):
as of this year.
But back then it was Nips.
Um, and basically it's, it'speople who really wanted to
focus more on the neurosciencerather than the math and pure
math and machine learning.
And so it's a great conferencebecause it's, um, it's grown a
lot and it's a really nice mixof experimental neuroscientists
and computationalneuroscientists getting together
(01:34):
in one place and discussing alot of different topics.
So it's one of the onlyconferences where you'll see
almost like a 50, 50 mix betweenexperimental papers and
computational papers.
Matt (01:45):
So how well does Numenta fit into that mix?
Where do we fit in there?
Subutai (01:51):
I think we're like the prototypical lab in some sense
because we, we interact a tonwith experimental
neuroscientists.
A lot of the stuff we do reallydraws on, on their work and
their research.
Um, and we have computationalmodels that take into account a
lot of biological details, so wealways get really great
reception at, uh, at COSYNEwhenever we present stuff.
Matt (02:13):
Great.
What did you find interestingthere since you attended this
year?
Subutai (02:17):
Yeah, this year was, um, uh, so this year it was in
Lisbon, Portugal, which isdifferent.
Uh, mostly it's been in SaltLake City or Denver, uh, you
know, in the rocky mountains,basically.
Matt (02:29):
Not very exotic.
Subutai (02:30):
Not very exotic and it's been hard for Europeans and
international people to getthere.
This year they made a point ofmentioning this.
It was a lot more internationaland uh, that was great.
And you could see the mix ofideas.
It was a little bit different.
Um, so that was pretty cool.
It's also, it wasn't nice to bein Lisbon, um, uh, in, in terms
of kind of what I found,interesting there.
(02:52):
There's a lot of stuff.
Um, uh, you know, Edvard Moserwas there and he talked about
their work on grid cells.
Um, and that was, we arefamiliar with a lot of that
work, but it was just great tosee him present and talk about
it.
So that was really cool.
Um, there was, uh, after themain conference there's
(03:12):
workshops and this year therewere two workshops that were
extremely relevant to our, ourwork.
So there was a full day workshopon continuous learning.
Matt (03:21):
A full day workshop, full day workshop mean you come bring
your computer and do actual workor you just sit and listen.
Subutai (03:26):
Yeah.
So the, these workshops are likemini conferences, so they're
split up.
So the COSYNE as a whole, hasabout a thousand attendees
roughly.
And each workshop had anywherebetween 20 to a hundred people
attend.
Matt (03:39):
That's pretty big.
Subutai (03:40):
It's still pretty big.
Um, but, uh, what you have is aset of talks and it tends to be
more discussion focused sopeople can sometimes interrupt
during a talk.
Um, and there's a lot more roomfor a discussion and you're
pretty much looking at one focustopic for the whole day.
So everyone there is kind ofinterested in that topic and
it's a nice way to meet people.
Matt (04:01):
Yeah.
Working on the same stuff thatyou're interested in.
Subutai (04:04):
Yeah, exactly.
Yeah.
And so it was nice to see awhole topic on continuous
learning.
Um, so again, they had someexperimental work showing how
people can learn or when theyforget and things like that.
And then some work in machinelearning that has tried to
address the problem.
So continuous learning,
Matt (04:21):
Is it the same as thing as online learning?
Would you say...
Subutai (04:23):
yeah, it's, it's, it's similar.
Yeah.
And some people call it lifelonglearning...
Matt (04:28):
Oh, I haven't heard that.
Subutai (04:30):
Um, so that was that.
I think I learned a lot fromthere.
So that was really cool.
And then the next day we had afull day workshop on coordinate
transformations and referenceframes.
Matt (04:40):
Oh Wow.
Subutai (04:41):
So that's of course extremely relevant.
Matt (04:43):
Grid cell heavy?
Subutai (04:44):
It was very grid cell and hippocampus heavy I'll say.
So it was a lot abouthippocampal stuff and a little
bit on the on Cortex.
Um, but there was a lot ofreally interesting kind of ideas
and, and it was just interestingto see how people think about,
uh, these, these terms.
And so, so,
Matt (05:01):
so, uh, did you attend just to see the, the workshops
and the presentations or did youpresent anything?
Subutai (05:07):
So, uh, we had a poster there so it was not just
attending but also presentingsome of our work and I can talk
about that and it's, uh, and youknow, that way you, there's
really a two way interactionbetween the, the other
conference attendees and ourstuff.
And um, so that's good.
Matt (05:24):
So that's where there's like a hall of posters and what
do you have like six feet byeight feet or something like
that to shove as muchinformation and graphics as you
can.
Subutai (05:33):
Um, I might have tweeted the picture of the
poster session, I forget, butit's, yeah, you have a large
hall and maybe 50 posters orsomething or, or a hundred
posters sometimes in, in a hall.
And you're separated by aboutfive feet or six feet.
And each poster is about, youknow, six by three years,
something like that.
And then you stand in front ofyour poster and anyone who's
(05:54):
interested can come by and talkto you about it.
Usually there's an open bar atthat time as well.
So that kind of lubricates thediscussion sometimes.
Um, and it's a great way toengage with people one on one or
one on two or, or so on andgoing into depth and, and, and
the topics.
And you also get to walk aroundand meet other people.
And I, in many ways, I like itbetter than the oral, the oral
(06:17):
sessions are also great, butthis is a completely different
type of interaction.
Matt (06:21):
You get a lot of one on one.
Subutai (06:22):
You got a lot of one on one.
Matt (06:23):
Meet people, press the flesh as they say.
Subutai (06:26):
Exactly.
Yeah.
Matt (06:27):
So what was your poster about?
Subutai (06:28):
Uh, so this year we had something slightly different.
We had a poster on the Thalamus.
Matt (06:35):
It's a hot topic.
Subutai (06:38):
It's a hot topic.
I did a blog post on it, on whymore people should pay attention
to the Thalamus.
And uh, the, so this was a kindof a novel proposal on how, uh,
the Thalamus, a dendriticmechanisms and the talents could
be used to make the Thalamus avery dynamic kind of routing and
multiplexing system.
Um, and so we walked through itin a fair amount of detail, how
(07:01):
exactly that might happen inthe, in the Thalamus.
Matt (07:04):
And you worked with a collaborator?
Subutai (07:06):
Yeah, so this was a collaboration with Carmen
Varella who was at MIT when westarted discussing and now she's
a professor at Florida AtlanticUniversity.
So she, and I had been workingon this since this was, this
actually came out of our guestsscientists program.
So she was at MIT and then shevisited us here for a few days
and we discussed all thingsthalamus.
(07:26):
And then, uh, this is one ofthose things where she was
talking about some specificmechanisms and I was thinking
about from a computationalstandpoint and we both kind of
realized, hey, we could, itcould be used to fulfill this
computational need that we thinkwe have.
And it could be kind of theperfect mechanism for that
because I think it's a reallygood example of experimental
(07:47):
theory collaboration.
Matt (07:48):
Yeah.
I think that's neat too.
To see a neuroscientist come andvisit and then we collaborate
with them and go present aconcept that's very, that's
perfect.
Subutai (07:56):
Yes.
And this was not my main job in,this was not her main job.
This sort of came out suddenlyout of this collaboration.
And so we both been kind ofworking on it on the side,
trying to develop the idea.
Matt (08:06):
and if you want to find out more about your Carmen
Varela and her work on thalamus,there's an interview that I did
with her, uh, last year thatI'll link in the podcast show
notes.
Um, so let's talk about some ofthe details of this paper if you
don't mind.
Uh, and our last podcast Italked to Jeff about, about the
Thalamus, and I'm going to talkto you about some different
things about the thalamus.
So in this poster, you talkabout the difference between the
(08:28):
modes that the thalamus might bein, one's called the burst and
tonic.
Can you describe what thosemodes are?
Subutai (08:34):
Yeah, I can, uh, maybe before I jump into that and
maybe I can just give like ahigh level picture of the
thalamus, we can dive into thatspecific question as well.
So the Thalamus has, um, uh, youknow, many different possible
roles.
And Jeff has, uh, ideas aboutmatrix cells and how they can be
involved in timing and alsocontrolling of oscillations and
(08:55):
stuff.
And that's one type of cells.
Um, but in general, the Thalamusis right in between all cortical
regions.
So whenever a cortical regionprojects information forward to
another cortical region, there'ssort of a copy of the signal
that gets, uh, or not a copy ofthat exact same, but there's
also a pathway through theThalamus.
(09:16):
And for a long time peoplethought that the Thalamus was
just a gating center.
It would just relay informationfrom one part of the cortex or
the other.
Matt (09:25):
These cells are called relay cells.
Subutai (09:25):
They're called relay cells exactly because of that.
But if you look at the anatomyof the Thalamus and the
physiology, it looks a lot morecomplicated than that.
And it can't just be a, a relaystation.
And the other thing is that theconnections between the cortex
and the Thalamus are highlyregular.
They're somewhat complicated,but they are very regular.
(09:47):
So, just like the corticalcolumn has a regular structure,
the cortical column, Thalamicinteractions, and the anatomy
there has a very regularstructure.
And this has been written aboutby Murray Sherman and Ray
Guillery and others.
There's a very regular kind ofpattern.
Matt (10:03):
Like across cortex
Subutai (10:04):
Across cortex.
Matt (10:06):
Across neocortex.
I think Jeff said this, said itin a way, like it's almost like
it's a part of the corticalcolumn circuit that the thalamus
could be another layer.
Subutai (10:15):
And that's that.
And that's, yeah, I think thatcomes from me, but yeah, cause
I, I, the way I see it, I justthink of the thalamus as layer
seven of the cortex.
It's so tightly tied to theother layers.
Matt (10:27):
That decomplicates it a bit.
Subutai (10:29):
Yeah.
Yeah.
I think it makes it easier.
Um, and it gives you a filter ofhow to read through thalamic
papers as well when you, whenyou read it.
And so for us, uh, doing modelsof the cortical column, I think
it's critical to understand thethalamus and how it, how it
operates.
Matt (10:42):
So what are these modes that it might not?
Subutai (10:44):
Yeah, so thalamic cells, uh, these relay cells or
thalama-cortical cells, uh, theyproject to the cortex.
Um, and they, uh, cortical cellsare, you know, the fire and they
don't fire.
Um, there's an but, uh, thalamiccells, they tend to have a much
higher regular rate of firingthen cortical cells.
(11:06):
Um, but what's really importantwith, uh, with thalamic cells is
not just whether they fire ornot, but what mode they're in
when they're firing.
And the two main modes are tonicmode and burst mode.
And tonic mode is kind of theregular neuron firing.
It Just fires at a regular rateor you know, at different rates,
but it's just sort of regularlyfiring in.
(11:26):
Burst mode is this special modewhere it's quiet for a bit and
then there's a burst of spikesat a very high frequency that,
that just gets triggered.
Um, and some people think thishas very different downstream
implications.
So if you think about a corticalneuron and your thalamic input
is sort of, you know, regularlyfiring you treat it one way and
(11:47):
if it's bursting, you treat itdifferently.
It has a different effect onthe, on the post synaptic cell.
Matt (11:53):
so said you said bursting, is that equivalent to the idea
of bursting in cortex as well?
Subutai (11:58):
So there's bursts.
Um, there's burst type activityin cortex as well.
And we've used the term in acouple of different ways.
So, um, so if you're familiarwith our temporal memory, we
talk about columns bursting.
Matt (12:10):
Minicolumns bursting.
Subutai (12:10):
Mini columns bursting.
Thank you.
Uh, and that's a slightlydifferent thing.
That's more like, oh, there's aburst of activity across many
cells.
So this is a single cell thatfires at a very fast rate for
just like 50 milliseconds or so.
Okay.
And that also happens in theneocortex.
So layer five cells for example,are known to do bursting in, in
(12:34):
certain situations.
And I think they've seen it inother cells as well.
Matt (12:37):
So when we talk about the modes of the Thalamus, tonic
versus burst, is this, are wetalking about individual cells
and, and being in differentmodes in their surrounding cells
or are whole sections in onemode or the other?
Subutai (12:47):
Uh, so we think it can be both, but the property is a
property of a single cell.
Okay.
Um, but we think what would wepresented our poster is that the
cortex can control large partsof Thalamus and, uh, all of
thalamus until this, this partneeds to be in tonic mode and
this part needs to be in burstmountain and you can kind of
(13:08):
think of burst mode is like, uh,uh, pay attention to me mode
like this, it has a strongerimpact or, you know, it's, it's
a different signal than, thanthe tonic mode.
Matt (13:19):
Yeah.
Um, so how does the cortex haveany control over those cells and
what mode they're in?
Subutai (13:26):
Yeah.
So thalamocortical cells, orthese relay cells, they project
to cortex.
But what's interesting isthere's a feedback projection
from cortex as well.
Matt (13:35):
There's a loop there.
Subutai (13:36):
There's, uh, it's, it's different cells that project
back.
Matt (13:39):
Oh, okay.
Subutai (13:39):
So, uh, it's so typically Thalmus might project
to Layer 4 and some parts ofLayer 5.
Um, and then Layer 6A in cortexprojects back to the Thalamus.
Okay.
Um, and these are the corticalthalamic cells.
The terminology is a little, uh,uh, sometimes confusing.
SoLayer 6A projects back to theThalamus.
(14:02):
And the projection pattern backis kind of interesting and
intricate.
And what we proposed in thisposter is that Layer 6A cells
when they project back to theThalamus, they can very
precisely control which cellsare in tonic mode or burst mode,
uh, in a, in a very, veryprecise way.
Matt (14:19):
Based on its representation of whatever is
being modeled.
within the column?
Subutai (14:22):
Yeah, yeah, exactly.
So the other parts, other layersof cortex could be modeling
something.
And, um, you know, and thenLayer 6A could very precisely
control what's going on in howthe Thalamus, uh, behaves
basically.
And what's interesting is thatthe way Layer 6A projects to the
Thalamus.
Again, this is very regularpattern, um, and it projects to
(14:46):
two different cell types in thethalamus.
So one is this, um, what'scalled the thalamicreticular
nucleus, which is a layer ofinhibitory cells that surrounds
the Thalamus.
And then it also directlyprojects to the relay cells to
the thalama-cortical cells.
And um, so Layer 6A projects toboth of those.
(15:09):
And then the inhibitory cells,these trn cells, they also
projected the dendrites of theserelay cells.
Matt (15:17):
So there's two stops.
Subutai (15:18):
There's two stops there.
And so it's kind of ainteresting pattern.
It's a regular pattern.
And so what we realized is thatthe way the dendrites function,
um, on, on these relay cells,and there's this thing called
these t calcium channels, whichI can describe, I don't know how
much detail to go into here, butuh, but um, the, the way these
(15:40):
dendrites function, um, Layer 6Acould be sending very specific
codes down.
These trn sales could berecognizing these codes and then
causing the relay cells to getinto burst mode versus tonic
mode through these, through thisdendritic mechanism.
Matt (15:57):
And that's not even the direct information they're
sending to the relay cells.
That's like a contextual signal?
Subutai (16:04):
Yeah.
So imagine like the trn cell isdetecting context of different
types and, um, recognizes thatcontext and then sends these
inhibitory signals, uh, to thedendrites of the relay cells,
right?
And why inhibitory signals?
Well, these, uh, the dendritesof the relay cells, they're
sprinkled with these, what'scalled these t type calcium
(16:24):
channels.
These are voltage gated channelsand they have a very, very
particular behavior.
Um, they have sort of threestages.
Uh, they can be in a normalinactivated stage.
And if these channels are in aninactivated stage, then whenever
the relay cell fires that justfires, it's like tonic mode,
right?
Um, but if you inhibit thedendrite, um, and these T type
(16:50):
calcium channels, if you give ithyperpolarizing input for a
period of time, they get intothis de-inactivated state.
Don't ask me who made up theseterms.
That's the term, de-inactivatedstate.
And what, what happens is thatin that state, if the cell
fires, if it gets input andfires, it's going to then
trigger a burst.
(17:10):
What happens is these calciumchannels open up and the, uh,
short flood of calcium comesinto the cell and it triggers
multiple spikes.
Matt (17:19):
And so it's a little chain reaction going gets a little
Twitter, Twitter, Twitter.
Subutai (17:22):
Exactly.
So this is very particular, youknow, three stage thing.
It's in tonic mode or it'sready, it's de-inactivated,
which I think of as ready toburst and then it's bursting.
Um, and so this trn cells cancontrol this.
It can switch the dendrites, um,you know, between these
different modes, um, to causethe cell to be burst ready or
(17:45):
tonic mode.
Matt (17:45):
So the thing that since there's, there's, um, routes
going from the Cortex to theThalamus and from the Thalamus
to the Cortex, like what'scontrolling what?
Like, uh, the cortex isobviously affecting the thalamus
and what mode it's going to be.
Yeah.
And then what is the Thalamusthen sending back up the cortex?
Subutai (18:02):
Okay.
So the Thalamus is sendingwhatever input it's getting from
some other source, and then thisLayer 6A, uh, signal is sort of
modulating or controlling thenature, the way it sends that
input.
Okay.
Uh, so think the easiest way tothink about it is if you look at
the visual Cortex, the firststage, um, the thalamic center
(18:24):
is called Lgn or lateralgeniculate nucleus, it gets
input through the retina.
And the, um, one of the uses ofthis mechanism might be as an
attentional mechanism.
So the layer six AI or cortexwould be saying, hey, I really
want to pay interest, uh, uh, tothis part of the retina.
And what it'll do is it'll setthose cells to be in burst ready
(18:46):
mode through this, um, throughthis top down code.
And now, whenever, whateverinput comes in to the retina,
now this, the information that'swithin the focus of attention
will burst and may have a biggerimpact on learning, on just
inference.
Um, both actually and the stuffoutside your focus of attention
might have a lower number.
Matt (19:07):
So, so, so, so the Thalamus then has a, has a way
to sort of focus attention onsome aspect of the sensory field
and put and have more emphasison that part of it.
Subutai (19:18):
Exactly.
Exactly.
Yeah.
And because it's dendritic, itcan be, it's down at the level
of individual cells.
Um, and it can, so it can be avery, very precise, uh, control
of it.
Uh, another possible thing coulddo is be filtering the content.
Um, so if, you know, we knowthat the retina, uh, is not just
like pixels in a camera.
(19:39):
It's actually detecting featureslike motion and you know, its
color and so on.
And, uh, cortex could be tellingthe Thalamus, Hey, I'm only
interested in things that aremoving away from me or coming
towards me.
Um, and so would emphasize thosesignals and de-emphasize others.
Matt (19:58):
Oh so that could be like how the Cortex is affecting what
the Thalamus is doing byproviding information about what
should be attended to.
Subutai (20:04):
Exactly.
Yeah.
Yeah.
So it gives a tremendous amountof power into, you know, how you
kind of filter the stuff comingin.
It's not just a relay cell.
You can really contextuallyfilter it.
Um, and again, because it'sdendritic, it can actually have
a very powerful, uh, it can dolots of transformations of the
(20:25):
input.
Um, if you're a computerscientist, you might know about
multiplexers where you set aparticular switching function.
Depending on a state, you setthe system to switch input in a
certain way.
And when input comes in and getsmoved around and switched
around, so the thalamus could dothat via control of the cortex.
Matt (20:47):
That's one of the hypotheses.
Subutai (20:48):
That's one of the hypotheses.
So it's actually a very powerfulway of kind of transforming the
inputs, um, uh, that are comingin.
Matt (20:56):
What could the brain be doing with that?
Subutai (21:00):
Yeah, that's a great question.
Um, so, uh, you know, by talkingabout filtering and attention,
uh, Jeff has talked about waysof doing variance in there, so
it could actually beimplementing a little bit of a
translation and variants orscale invariants.
Um, it could be shifting inputsleft or right, and there's some
(21:20):
theories on how specific typesof attention might be
implemented with shifting, uh,with what's called shifter
circuits.
Um, so there's a lot ofdifferent possibilities.
And in this poster we're justsaying, hey, this is a really
powerful mechanism of doing allsorts of transformations and
then we can figure out exactlywhat the cortex might want to do
(21:42):
with this.
And what's cool is that it's notjust LGN to V1.
This is between every singlecortical region.
The thalamus is a center routinginformation or transforming
information between all corticalregions in some sense.
Matt (21:57):
Yeah.
Talking to Jeff in the lastpodcast made me realize that, I
mean, you can't exist withoutyour Thalamus, the neocortex
can't do anything without it andvice versa.
The thalamus doesn't, can't doanything without the model the
neocortex provides, too.
Subutai (22:11):
Exactly.
They're intricately tiedtogether.
Matt (22:13):
Yeah.
That was his point is I think ofit as the same thing at this
point.
And that makes sense to me.
So it's really important that wefigure out what it's doing.
Subutai (22:20):
Yeah.
It's, it's, it's part offiguring out what the cortical
column is doing.
It's no different.
Matt (22:25):
Well, that's what we're working on here at Numenta.
And, uh, thanks Subutai forsharing, uh, your, your COSYNE
experience with the podcastviewers.
Subutai (22:33):
Thank you.
This was fun.
Matt (22:34):
It's been a pleasure.
Thanks for listening to theNumenta On Intelligence podcast.
Next episode, I'll talk again toJeff Hawkins.
We'll be talking about thedefinition of intelligence.
Until next time.
Today on the Numenta On Intelligence podcast.
Subutai (00:04):
If you're a computer scientist, you might know about
multiplexers where you set aparticular switching function.
Depending on a state, you, youset the system to switch input
in a certain way and when inputcomes in and gets moved around
and switched around, so theBallon boundless could do that
through by control of the viacontrol of the neocortex.
Matt (00:24):
That was Subutai Ahmad on the Thalamus.
I'm Matt Taylor, Numentacommunity manager, and you're
listening to the Numenta OnIntelligence podcast.
I'm here with, Subutai Ahmad ourVP of engineering at Numenta and
he recently attended the COSYNEconference and I wanted to talk
to you about it.
(00:44):
High.
Subutai.
Subutai (00:45):
Hi Matt.
Happy to be here.
Matt (00:47):
Awesome.
So, um, tell me about the COSYNEconference.
What, who attends and what is itall about?
Subutai (00:52):
Yeah, so the COSYNE conference is pretty much the
main conference in computationalsystems, neuroscience.
That's what COSYNE stands for,every year.
And it's probably one of themain conferences that we attend
every year.
Uh, it originally split out fromNIPS, um, way back in the 90s.
I think, um, which is now called[inaudible], it's called Neurips
(01:12):
as of this year.
But back then it was Nips.
Um, and basically it's, it'speople who really wanted to
focus more on the neurosciencerather than the math and pure
math and machine learning.
And so it's a great conferencebecause it's, um, it's grown a
lot and it's a really nice mixof experimental neuroscientists
and computationalneuroscientists getting together
(01:34):
in one place and discussing alot of different topics.
So it's one of the onlyconferences where you'll see
almost like a 50, 50 mix betweenexperimental papers and
computational papers.
Matt (01:45):
So how well does Numenta fit into that mix?
Where do we fit in there?
Subutai (01:51):
I think we're like the prototypical lab in some sense
because we, we interact a tonwith experimental
neuroscientists.
A lot of the stuff we do reallydraws on, on their work and
their research.
Um, and we have computationalmodels that take into account a
lot of biological details, so wealways get really great
reception at, uh, at COSYNEwhenever we present stuff.
Matt (02:13):
Great.
What did you find interestingthere since you attended this
year?
Subutai (02:17):
Yeah, this year was, um, uh, so this year it was in
Lisbon, Portugal, which isdifferent.
Uh, mostly it's been in SaltLake City or Denver, uh, you
know, in the rocky mountains,basically.
Matt (02:29):
Not very exotic.
Subutai (02:30):
Not very exotic and it's been hard for Europeans and
international people to getthere.
This year they made a point ofmentioning this.
It was a lot more internationaland uh, that was great.
And you could see the mix ofideas.
It was a little bit different.
Um, so that was pretty cool.
It's also, it wasn't nice to bein Lisbon, um, uh, in, in terms
of kind of what I found,interesting there.
(02:52):
There's a lot of stuff.
Um, uh, you know, Edvard Moserwas there and he talked about
their work on grid cells.
Um, and that was, we arefamiliar with a lot of that
work, but it was just great tosee him present and talk about
it.
So that was really cool.
Um, there was, uh, after themain conference there's
(03:12):
workshops and this year therewere two workshops that were
extremely relevant to our, ourwork.
So there was a full day workshopon continuous learning.
Matt (03:21):
A full day workshop, full day workshop mean you come bring
your computer and do actual workor you just sit and listen.
Subutai (03:26):
Yeah.
So the, these workshops are likemini conferences, so they're
split up.
So the COSYNE as a whole, hasabout a thousand attendees
roughly.
And each workshop had anywherebetween 20 to a hundred people
attend.
Matt (03:39):
That's pretty big.
Subutai (03:40):
It's still pretty big.
Um, but, uh, what you have is aset of talks and it tends to be
more discussion focused sopeople can sometimes interrupt
during a talk.
Um, and there's a lot more roomfor a discussion and you're
pretty much looking at one focustopic for the whole day.
So everyone there is kind ofinterested in that topic and
it's a nice way to meet people.
Matt (04:01):
Yeah.
Working on the same stuff thatyou're interested in.
Subutai (04:04):
Yeah, exactly.
Yeah.
And so it was nice to see awhole topic on continuous
learning.
Um, so again, they had someexperimental work showing how
people can learn or when theyforget and things like that.
And then some work in machinelearning that has tried to
address the problem.
So continuous learning,
Matt (04:21):
Is it the same as thing as online learning?
Would you say...
Subutai (04:23):
yeah, it's, it's, it's similar.
Yeah.
And some people call it lifelonglearning...
Matt (04:28):
Oh, I haven't heard that.
Subutai (04:30):
Um, so that was that.
I think I learned a lot fromthere.
So that was really cool.
And then the next day we had afull day workshop on coordinate
transformations and referenceframes.
Matt (04:40):
Oh Wow.
Subutai (04:41):
So that's of course extremely relevant.
Matt (04:43):
Grid cell heavy?
Subutai (04:44):
It was very grid cell and hippocampus heavy I'll say.
So it was a lot abouthippocampal stuff and a little
bit on the on Cortex.
Um, but there was a lot ofreally interesting kind of ideas
and, and it was just interestingto see how people think about,
uh, these, these terms.
And so, so,
Matt (05:01):
so, uh, did you attend just to see the, the workshops
and the presentations or did youpresent anything?
Subutai (05:07):
So, uh, we had a poster there so it was not just
attending but also presentingsome of our work and I can talk
about that and it's, uh, and youknow, that way you, there's
really a two way interactionbetween the, the other
conference attendees and ourstuff.
And um, so that's good.
Matt (05:24):
So that's where there's like a hall of posters and what
do you have like six feet byeight feet or something like
that to shove as muchinformation and graphics as you
can.
Subutai (05:33):
Um, I might have tweeted the picture of the
poster session, I forget, butit's, yeah, you have a large
hall and maybe 50 posters orsomething or, or a hundred
posters sometimes in, in a hall.
And you're separated by aboutfive feet or six feet.
And each poster is about, youknow, six by three years,
something like that.
And then you stand in front ofyour poster and anyone who's
(05:54):
interested can come by and talkto you about it.
Usually there's an open bar atthat time as well.
So that kind of lubricates thediscussion sometimes.
Um, and it's a great way toengage with people one on one or
one on two or, or so on andgoing into depth and, and, and
the topics.
And you also get to walk aroundand meet other people.
And I, in many ways, I like itbetter than the oral, the oral
(06:17):
sessions are also great, butthis is a completely different
type of interaction.
Matt (06:21):
You get a lot of one on one.
Subutai (06:22):
You got a lot of one on one.
Matt (06:23):
Meet people, press the flesh as they say.
Subutai (06:26):
Exactly.
Yeah.
Matt (06:27):
So what was your poster about?
Subutai (06:28):
Uh, so this year we had something slightly different.
We had a poster on the Thalamus.
Matt (06:35):
It's a hot topic.
Subutai (06:38):
It's a hot topic.
I did a blog post on it, on whymore people should pay attention
to the Thalamus.
And uh, the, so this was a kindof a novel proposal on how, uh,
the Thalamus, a dendriticmechanisms and the talents could
be used to make the Thalamus avery dynamic kind of routing and
multiplexing system.
Um, and so we walked through itin a fair amount of detail, how
(07:01):
exactly that might happen inthe, in the Thalamus.
Matt (07:04):
And you worked with a collaborator?
Subutai (07:06):
Yeah, so this was a collaboration with Carmen
Varella who was at MIT when westarted discussing and now she's
a professor at Florida AtlanticUniversity.
So she, and I had been workingon this since this was, this
actually came out of our guestsscientists program.
So she was at MIT and then shevisited us here for a few days
and we discussed all thingsthalamus.
(07:26):
And then, uh, this is one ofthose things where she was
talking about some specificmechanisms and I was thinking
about from a computationalstandpoint and we both kind of
realized, hey, we could, itcould be used to fulfill this
computational need that we thinkwe have.
And it could be kind of theperfect mechanism for that
because I think it's a reallygood example of experimental
(07:47):
theory collaboration.
Matt (07:48):
Yeah.
I think that's neat too.
To see a neuroscientist come andvisit and then we collaborate
with them and go present aconcept that's very, that's
perfect.
Subutai (07:56):
Yes.
And this was not my main job in,this was not her main job.
This sort of came out suddenlyout of this collaboration.
And so we both been kind ofworking on it on the side,
trying to develop the idea.
Matt (08:06):
and if you want to find out more about your Carmen
Varela and her work on thalamus,there's an interview that I did
with her, uh, last year thatI'll link in the podcast show
notes.
Um, so let's talk about some ofthe details of this paper if you
don't mind.
Uh, and our last podcast Italked to Jeff about, about the
Thalamus, and I'm going to talkto you about some different
things about the thalamus.
So in this poster, you talkabout the difference between the
(08:28):
modes that the thalamus might bein, one's called the burst and
tonic.
Can you describe what thosemodes are?
Subutai (08:34):
Yeah, I can, uh, maybe before I jump into that and
maybe I can just give like ahigh level picture of the
thalamus, we can dive into thatspecific question as well.
So the Thalamus has, um, uh, youknow, many different possible
roles.
And Jeff has, uh, ideas aboutmatrix cells and how they can be
involved in timing and alsocontrolling of oscillations and
(08:55):
stuff.
And that's one type of cells.
Um, but in general, the Thalamusis right in between all cortical
regions.
So whenever a cortical regionprojects information forward to
another cortical region, there'ssort of a copy of the signal
that gets, uh, or not a copy ofthat exact same, but there's
also a pathway through theThalamus.
(09:16):
And for a long time peoplethought that the Thalamus was
just a gating center.
It would just relay informationfrom one part of the cortex or
the other.
Matt (09:25):
These cells are called relay cells.
Subutai (09:25):
They're called relay cells exactly because of that.
But if you look at the anatomyof the Thalamus and the
physiology, it looks a lot morecomplicated than that.
And it can't just be a, a relaystation.
And the other thing is that theconnections between the cortex
and the Thalamus are highlyregular.
They're somewhat complicated,but they are very regular.
(09:47):
So, just like the corticalcolumn has a regular structure,
the cortical column, Thalamicinteractions, and the anatomy
there has a very regularstructure.
And this has been written aboutby Murray Sherman and Ray
Guillery and others.
There's a very regular kind ofpattern.
Matt (10:03):
Like across cortex
Subutai (10:04):
Across cortex.
Matt (10:06):
Across neocortex.
I think Jeff said this, said itin a way, like it's almost like
it's a part of the corticalcolumn circuit that the thalamus
could be another layer.
Subutai (10:15):
And that's that.
And that's, yeah, I think thatcomes from me, but yeah, cause
I, I, the way I see it, I justthink of the thalamus as layer
seven of the cortex.
It's so tightly tied to theother layers.
Matt (10:27):
That decomplicates it a bit.
Subutai (10:29):
Yeah.
Yeah.
I think it makes it easier.
Um, and it gives you a filter ofhow to read through thalamic
papers as well when you, whenyou read it.
And so for us, uh, doing modelsof the cortical column, I think
it's critical to understand thethalamus and how it, how it
operates.
Matt (10:42):
So what are these modes that it might not?
Subutai (10:44):
Yeah, so thalamic cells, uh, these relay cells or
thalama-cortical cells, uh, theyproject to the cortex.
Um, and they, uh, cortical cellsare, you know, the fire and they
don't fire.
Um, there's an but, uh, thalamiccells, they tend to have a much
higher regular rate of firingthen cortical cells.
(11:06):
Um, but what's really importantwith, uh, with thalamic cells is
not just whether they fire ornot, but what mode they're in
when they're firing.
And the two main modes are tonicmode and burst mode.
And tonic mode is kind of theregular neuron firing.
It Just fires at a regular rateor you know, at different rates,
but it's just sort of regularlyfiring in.
(11:26):
Burst mode is this special modewhere it's quiet for a bit and
then there's a burst of spikesat a very high frequency that,
that just gets triggered.
Um, and some people think thishas very different downstream
implications.
So if you think about a corticalneuron and your thalamic input
is sort of, you know, regularlyfiring you treat it one way and
(11:47):
if it's bursting, you treat itdifferently.
It has a different effect onthe, on the post synaptic cell.
Matt (11:53):
so said you said bursting, is that equivalent to the idea
of bursting in cortex as well?
Subutai (11:58):
So there's bursts.
Um, there's burst type activityin cortex as well.
And we've used the term in acouple of different ways.
So, um, so if you're familiarwith our temporal memory, we
talk about columns bursting.
Matt (12:10):
Minicolumns bursting.
Subutai (12:10):
Mini columns bursting.
Thank you.
Uh, and that's a slightlydifferent thing.
That's more like, oh, there's aburst of activity across many
cells.
So this is a single cell thatfires at a very fast rate for
just like 50 milliseconds or so.
Okay.
And that also happens in theneocortex.
So layer five cells for example,are known to do bursting in, in
(12:34):
certain situations.
And I think they've seen it inother cells as well.
Matt (12:37):
So when we talk about the modes of the Thalamus, tonic
versus burst, is this, are wetalking about individual cells
and, and being in differentmodes in their surrounding cells
or are whole sections in onemode or the other?
Subutai (12:47):
Uh, so we think it can be both, but the property is a
property of a single cell.
Okay.
Um, but we think what would wepresented our poster is that the
cortex can control large partsof Thalamus and, uh, all of
thalamus until this, this partneeds to be in tonic mode and
this part needs to be in burstmountain and you can kind of
(13:08):
think of burst mode is like, uh,uh, pay attention to me mode
like this, it has a strongerimpact or, you know, it's, it's
a different signal than, thanthe tonic mode.
Matt (13:19):
Yeah.
Um, so how does the cortex haveany control over those cells and
what mode they're in?
Subutai (13:26):
Yeah.
So thalamocortical cells, orthese relay cells, they project
to cortex.
But what's interesting isthere's a feedback projection
from cortex as well.
Matt (13:35):
There's a loop there.
Subutai (13:36):
There's, uh, it's, it's different cells that project
back.
Matt (13:39):
Oh, okay.
Subutai (13:39):
So, uh, it's so typically Thalmus might project
to Layer 4 and some parts ofLayer 5.
Um, and then Layer 6A in cortexprojects back to the Thalamus.
Okay.
Um, and these are the corticalthalamic cells.
The terminology is a little, uh,uh, sometimes confusing.
SoLayer 6A projects back to theThalamus.
(14:02):
And the projection pattern backis kind of interesting and
intricate.
And what we proposed in thisposter is that Layer 6A cells
when they project back to theThalamus, they can very
precisely control which cellsare in tonic mode or burst mode,
uh, in a, in a very, veryprecise way.
Matt (14:19):
Based on its representation of whatever is
being modeled.
within the column?
Subutai (14:22):
Yeah, yeah, exactly.
So the other parts, other layersof cortex could be modeling
something.
And, um, you know, and thenLayer 6A could very precisely
control what's going on in howthe Thalamus, uh, behaves
basically.
And what's interesting is thatthe way Layer 6A projects to the
Thalamus.
Again, this is very regularpattern, um, and it projects to
(14:46):
two different cell types in thethalamus.
So one is this, um, what'scalled the thalamicreticular
nucleus, which is a layer ofinhibitory cells that surrounds
the Thalamus.
And then it also directlyprojects to the relay cells to
the thalama-cortical cells.
And um, so Layer 6A projects toboth of those.
(15:09):
And then the inhibitory cells,these trn cells, they also
projected the dendrites of theserelay cells.
Matt (15:17):
So there's two stops.
Subutai (15:18):
There's two stops there.
And so it's kind of ainteresting pattern.
It's a regular pattern.
And so what we realized is thatthe way the dendrites function,
um, on, on these relay cells,and there's this thing called
these t calcium channels, whichI can describe, I don't know how
much detail to go into here, butuh, but um, the, the way these
(15:40):
dendrites function, um, Layer 6Acould be sending very specific
codes down.
These trn sales could berecognizing these codes and then
causing the relay cells to getinto burst mode versus tonic
mode through these, through thisdendritic mechanism.
Matt (15:57):
And that's not even the direct information they're
sending to the relay cells.
That's like a contextual signal?
Subutai (16:04):
Yeah.
So imagine like the trn cell isdetecting context of different
types and, um, recognizes thatcontext and then sends these
inhibitory signals, uh, to thedendrites of the relay cells,
right?
And why inhibitory signals?
Well, these, uh, the dendritesof the relay cells, they're
sprinkled with these, what'scalled these t type calcium
(16:24):
channels.
These are voltage gated channelsand they have a very, very
particular behavior.
Um, they have sort of threestages.
Uh, they can be in a normalinactivated stage.
And if these channels are in aninactivated stage, then whenever
the relay cell fires that justfires, it's like tonic mode,
right?
Um, but if you inhibit thedendrite, um, and these T type
(16:50):
calcium channels, if you give ithyperpolarizing input for a
period of time, they get intothis de-inactivated state.
Don't ask me who made up theseterms.
That's the term, de-inactivatedstate.
And what, what happens is thatin that state, if the cell
fires, if it gets input andfires, it's going to then
trigger a burst.
(17:10):
What happens is these calciumchannels open up and the, uh,
short flood of calcium comesinto the cell and it triggers
multiple spikes.
Matt (17:19):
And so it's a little chain reaction going gets a little
Twitter, Twitter, Twitter.
Subutai (17:22):
Exactly.
So this is very particular, youknow, three stage thing.
It's in tonic mode or it'sready, it's de-inactivated,
which I think of as ready toburst and then it's bursting.
Um, and so this trn cells cancontrol this.
It can switch the dendrites, um,you know, between these
different modes, um, to causethe cell to be burst ready or
(17:45):
tonic mode.
Matt (17:45):
So the thing that since there's, there's, um, routes
going from the Cortex to theThalamus and from the Thalamus
to the Cortex, like what'scontrolling what?
Like, uh, the cortex isobviously affecting the thalamus
and what mode it's going to be.
Yeah.
And then what is the Thalamusthen sending back up the cortex?
Subutai (18:02):
Okay.
So the Thalamus is sendingwhatever input it's getting from
some other source, and then thisLayer 6A, uh, signal is sort of
modulating or controlling thenature, the way it sends that
input.
Okay.
Uh, so think the easiest way tothink about it is if you look at
the visual Cortex, the firststage, um, the thalamic center
(18:24):
is called Lgn or lateralgeniculate nucleus, it gets
input through the retina.
And the, um, one of the uses ofthis mechanism might be as an
attentional mechanism.
So the layer six AI or cortexwould be saying, hey, I really
want to pay interest, uh, uh, tothis part of the retina.
And what it'll do is it'll setthose cells to be in burst ready
(18:46):
mode through this, um, throughthis top down code.
And now, whenever, whateverinput comes in to the retina,
now this, the information that'swithin the focus of attention
will burst and may have a biggerimpact on learning, on just
inference.
Um, both actually and the stuffoutside your focus of attention
might have a lower number.
Matt (19:07):
So, so, so, so the Thalamus then has a, has a way
to sort of focus attention onsome aspect of the sensory field
and put and have more emphasison that part of it.
Subutai (19:18):
Exactly.
Exactly.
Yeah.
And because it's dendritic, itcan be, it's down at the level
of individual cells.
Um, and it can, so it can be avery, very precise, uh, control
of it.
Uh, another possible thing coulddo is be filtering the content.
Um, so if, you know, we knowthat the retina, uh, is not just
like pixels in a camera.
(19:39):
It's actually detecting featureslike motion and you know, its
color and so on.
And, uh, cortex could be tellingthe Thalamus, Hey, I'm only
interested in things that aremoving away from me or coming
towards me.
Um, and so would emphasize thosesignals and de-emphasize others.
Matt (19:58):
Oh so that could be like how the Cortex is affecting what
the Thalamus is doing byproviding information about what
should be attended to.
Subutai (20:04):
Exactly.
Yeah.
Yeah.
So it gives a tremendous amountof power into, you know, how you
kind of filter the stuff comingin.
It's not just a relay cell.
You can really contextuallyfilter it.
Um, and again, because it'sdendritic, it can actually have
a very powerful, uh, it can dolots of transformations of the
(20:25):
input.
Um, if you're a computerscientist, you might know about
multiplexers where you set aparticular switching function.
Depending on a state, you setthe system to switch input in a
certain way.
And when input comes in and getsmoved around and switched
around, so the thalamus could dothat via control of the cortex.
Matt (20:47):
That's one of the hypotheses.
Subutai (20:48):
That's one of the hypotheses.
So it's actually a very powerfulway of kind of transforming the
inputs, um, uh, that are comingin.
Matt (20:56):
What could the brain be doing with that?
Subutai (21:00):
Yeah, that's a great question.
Um, so, uh, you know, by talkingabout filtering and attention,
uh, Jeff has talked about waysof doing variance in there, so
it could actually beimplementing a little bit of a
translation and variants orscale invariants.
Um, it could be shifting inputsleft or right, and there's some
(21:20):
theories on how specific typesof attention might be
implemented with shifting, uh,with what's called shifter
circuits.
Um, so there's a lot ofdifferent possibilities.
And in this poster we're justsaying, hey, this is a really
powerful mechanism of doing allsorts of transformations and
then we can figure out exactlywhat the cortex might want to do
(21:42):
with this.
And what's cool is that it's notjust LGN to V1.
This is between every singlecortical region.
The thalamus is a center routinginformation or transforming
information between all corticalregions in some sense.
Matt (21:57):
Yeah.
Talking to Jeff in the lastpodcast made me realize that, I
mean, you can't exist withoutyour Thalamus, the neocortex
can't do anything without it andvice versa.
The thalamus doesn't, can't doanything without the model the
neocortex provides, too.
Subutai (22:11):
Exactly.
They're intricately tiedtogether.
Matt (22:13):
Yeah.
That was his point is I think ofit as the same thing at this
point.
And that makes sense to me.
So it's really important that wefigure out what it's doing.
Subutai (22:20):
Yeah.
It's, it's, it's part offiguring out what the cortical
column is doing.
It's no different.
Matt (22:25):
Well, that's what we're working on here at Numenta.
And, uh, thanks Subutai forsharing, uh, your, your COSYNE
experience with the podcastviewers.
Subutai (22:33):
Thank you.
This was fun.
Matt (22:34):
It's been a pleasure.
Thanks for listening to theNumenta On Intelligence podcast.
Next episode, I'll talk again toJeff Hawkins.
We'll be talking about thedefinition of intelligence.
Until next time.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Intentionally Disturbing
Join me on this podcast as I navigate the murky waters of human behavior, current events, and personal anecdotes through in-depth interviews with incredible people—all served with a generous helping of sarcasm and satire. After years as a forensic and clinical psychologist, I offer a unique interview style and a low tolerance for bullshit, quickly steering conversations toward depth and darkness. I honor the seriousness while also appreciating wit. I’m your guide through the twisted labyrinth of the human psyche, armed with dark humor and biting wit.