Will We Know When AI Becomes Conscious? The Epistemic Fog of Artificial Minds | Eric Schwitzgebel - Mind-Body Solution

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:08):
Hey, thanks so much for joining me one more time.
Yeah, it's a pleasure to be here.
Great to have you while while westart, I think it's best to just
let everyone know. Today we'll be chatting about
your upcoming book AI and Consciousness, and you recently
posted something online where you requested that anyone who
emails you any comments on the entire manuscript will be

(00:29):
thanked in print and receive an appreciatively signed hard copy.
So that's right, anything you want to say?
About it I I realized I need a clarification because a a couple
people read the manuscript and then sent me like a couple
paragraphs of here are some general thoughts and that might
not be quite enough for a signedcopy of the manuscript.

(00:51):
I wouldn't call that comments onthe entire manuscript.
Yeah. So be looking for a little more
specific than that to get the copy, but anybody who says any
comments at all will be thanked.I really do appreciate those
comments. I like getting comments from
across the spectrum of people's knowledge because I want it to

(01:13):
be accessible to people like your listeners who are
intellectually interested and engaged, but not specialists.
And I also want specialists to get value from it and people who
are specialists in fields that are adjacent.
So I value the comments from from everybody.
And and Eric preferably from mid-december.

(01:33):
I think that's the cut off you want.
That's right. I will be doing a full revision
starting in mid-december and then hopefully getting it to the
press in early 2026. So that's the call to action.
Everybody please give some well thought out comments as much as
possible. All right, Eric, let's begin.

(01:53):
I think wonderful book, by the way.
I think anyone who listens, who reads it, sorry, and listens to
this podcast will get a really great understanding of this
topic. AI and consciousness clearly
making the rounds every every time you go online.
This is the talk of the town in your book.
You start off immediately by framing this question.
You start to frame what we call the fog.

(02:15):
I think your opening line is allis fog.
What exactly is this epistemic fog and and why do you think we
won't know if AI becomes conscious until we've built
millions of disputably conscioussystems?
Yes, right. Well, that's the the whole book
is basically my answer to that to that question.

(02:36):
So condensing it briefly and I'msure we'll get into more detail
on this. The science of consciousness is
in its infancy. There are huge debates about the
most basic things. The range of opinions is vast,
the methodologies are highly contentious.

(02:59):
And even for entities who are a lot like us non human animals,
there's not a lot of consensus about whether they're can't
conscious or how you measure it.Though on the one hand it's
going to take a long time for the conscious consciousness
science that I think reach maturity if ever it does.

(03:20):
And at the same time I think we are on the cusp. 5 to 30 years
is my prediction of creating systems that substantial group
of really mainstream respectablescientific theories say these
things are conscious, they've got experiences, they're just as
capable of suffering and pleasure as you and I are and

(03:44):
therefore they deserve serious moral consideration.
Maybe we could call them persons.
They deserve rights or human rights or human like rights.
And at the same time, I think it's very plausible that there
will be a substantial number of mainstream scientific and
philosophical views that say, hey, look, these entities are

(04:05):
just shadow play of empty mimicry.
They have no more personhood or experience than a ceiling fan.
And we won't know which theory is right.
And people have opinions. Those opinions will be based in
scientific theories of one sort or another, or based in their
own personal inclinations and biases or their culture or their

(04:27):
religion or whatever. And those opinions will diverge.
And as a group, we won't know what to do, whether we're facing
real persons with real rights, serious rights, like rights to
reproduce, rights to a path to citizenship and the vote and the
rights to employment for money, or whether these are just like

(04:50):
our ordinary computers today anddon't deserve anything like that
kind of moral consideration, right.
So I think we're on the cusp of that kind of, I want to call it
a likely crisis. I think the purpose for this
episode for me, Eric, was to to make sure people read the book
because this episode will cover the book.

(05:12):
However, we want them to still engage and interact.
So the main goal here is we're going to take them on the
journey through the book and hopefully this inspires them to
comment even further and allows you to clarify your thoughts
even more. So that being said, is the fog
primarily empirical, IE let's say neuroscience and AI opacity
or or is it conceptual, like ourgrip on consciousness itself?

(05:35):
So we do. The two bleed together.
So let me say one thing I think it's not which a lot of people
might think it is, which is the the last thing you said, which
is that the problem is understanding the concept of

(05:56):
consciousness itself. I do think that people mean
different things by the word consciousness.
And sometimes people talk past each other when they talk about
consciousness, but I think thereis a primary sense of
consciousness that's used by consciousness scientists and by
philosophers that we can pretty much all agree on and is pretty

(06:23):
obvious. And that is really the target of
this. And I don't think that these
debates can be deemed primarily as confusion about that concept.
So to kind of help your listeners get a sense of this
concept that I'm referring to sometimes called phenomenal
consciousness in the literature,but I don't, I think the word

(06:45):
phenomenal is not necessary. It just helps disambiguate if
someone's confused about exactlywhat the reference is.
So I'll just use the word consciousness for this, right?
So consciousness is an obvious property that several things
that your listeners will be familiar with share, right?
So if your listeners think aboutthe sound of my voice as they're

(07:06):
hearing it right now, their experience of that.
If they think about their, say, a sudden rush of fear that they
can recall from the recent past,maybe when they almost got in an
accident or something like that.If they close their eyes and
form an image, if they can, of say, their mother's face.

(07:32):
If they pause and think about, OK, what's the best way to get
across town to grandma's house during rush hour, they think
about if you stub your toe, there's an experience of pain,
right? All of these things, the
imagery, that kind of episode ofthinking, inner speech, pain,

(07:52):
sensory experiences like auditory experiences and visual
experiences. They all have I think an obvious
property in common that is kind of used by saying they're
conscious or their experiences and other mental episodes don't
have that. So if you think about your
knowledge that Obama was U.S. President in 2010, which I

(08:16):
assume you weren't thinking about 5 minutes ago, 5 minutes
ago, that was an aspect of your mentality, your preparedness to
answer that question in a certain way.
But it wasn't any part of your experience, right?
If you think about the myelination of your axons or if
you think about skills that you have that you know are not being
used at the time, there are lotsof aspects of our mentality that

(08:36):
aren't part of our stream of experience.
This term consciousness really refers to that thing, which I
think is pretty obvious when youthink about it, that we all
share, though that's the target of philosophical and scientific
inquiry in the primary sense though.

(08:57):
So I don't think disagreements are really about primarily about
the reference of that term. The question is that thing, the
obvious thing that I have and you have and your listeners
have, is that going to be present in an AI system and to
what degree? That's the fundamental question

(09:20):
that is extremely difficult to answer scientifically.
You So at some point you have tothink if neither introspection
nor conceptual analysis can settle the essential features of
consciousness. I mean, if these tools fail,
what really can? Right.

(09:44):
So one of the things that I do in the book early on is I
describe 10 possibly essential features of consciousness.
And I won't go through all 10 for your listeners here, but

(10:04):
some they're, they're things like subjectivity, right?
The idea that consciousness always involves a kind of a
sense of a self or a subject whois conscious.
Another possibly necessary feature of consciousness is
unity. The idea that if if two
conscious experiences are being had by the same subject, if

(10:28):
there is a subject at a time, then they kind of are joined or
unified in a certain way rather than being held entirely
separately. Another really important one is
access, right? The idea that a conscious
experience is usable to available for downstream
cognitive processes. It's the kind of thing that you

(10:49):
can remember, the kind of thing that you can report on, the kind
of thing that can guide action. And I guess I'll mention one
more important one, which is luminance.
Luminosity, the idea that when you have an experience, you're
always in some sense aware of having that experience.

(11:11):
You know that you have it, or atleast you're in a position to
know that you have it. So those would be 4 examples of
these 10 features that philosophers and scientists have
sometimes thought are present whenever consciousness is
present. So these are what I call
possibly essential feature. So I think one of the first
questions that a theorist of consciousness needs to figure

(11:36):
out in order to make progress isOK, which of these 10 features
really is essential, right? So if it's something like
luminosity, the capacity to selfrepresent yourself as conscious
or, or to have some kind of knowledge of your experience,
then you're not going to have anAI system that's conscious

(12:00):
unless it has some kind of self representational capacity, some
ability to think about in some sense what's going on in its own
mind, whatever that is though, right?
That already suggests certain kinds of structural features
that would be necessary for an AI system, right?

(12:21):
But if you don't think luminosity is essential, then
those features might not be essential, right?
Or unity, or take unity is another case, right?
If unity is essential, then disunified AI systems are going
to be out, They're not going to be conscious entities, at least
individual conscious entities. Whereas if unity is not
necessary, then you might have you no more disunified entities

(12:41):
who are capable of consciousness, right?
So there are going to be obviousstructural and engineering
constraints that follow from theessentiality of some of these
properties. So I think that's that's both
something close to a necessary starting point for thinking

(13:04):
about these issues and also already extremely difficult to
resolve. So you had mentioned that I
think that there it's not possible to introspectively or
conceptually resolve these questions about about possible

(13:28):
essentiality, right? So that's what philosophers will
tend to do. They'll tend to say, ah, when I
introspect, I discover that conscious experience is always
unified, or it always comes witha kind of representation of
itself. And well #1 I don't think our

(13:54):
introspections are actually all that reliable about these kinds
of structural features. I mean, my 2011 book was really
all about that. Lexities of Consciousness is the
title of that book. But kind of even more
fundamentally, there's a what I call the problem of the narrow

(14:14):
evidence base. We're really just one kind of
entity. So it might be that in US
experiences are always unified. I'm not sure whether that's
true. It might be that in US, when you
have a conscious experience, there's always some kind of
awareness of it as happening, but it doesn't follow that

(14:38):
that's the case for all entities.
One example I use in the book isI say it's like going to the
Freemasons Club and then pollingeverybody and discovering that
everybody's a Mason and then concluding that all people are
Freemasons, right? Which is obviously it's a biased

(14:59):
sample, right? So you don't want to generalize
just from what's universal amonghumans to what would be
universal among all conscious being, right?
So I think that's the problem with introspection.
I can talk in a bit about problems with conceptual
arguments, but I've been talkinga long time and I want to just

(15:19):
pause there. No, I actually think that's a
good idea. Let's by all means conceptual
analysis. What's what's what's wrong with
that? Right.
So the nice thing about conceptual analysis is that you
don't have to worry about a biased sample, right?
So an example of something that you can get through conceptual
analysis is bachelors are not married or quad quadrilateral

(15:47):
rectangles have four sides, right?
You don't need to kind of do a poll of bachelors to figure out,
Oh yeah, it looks like they're all unmarried or right or, or
look at a bunch of bunch of rectangles and say, Oh yeah,
well, all rectangles have four sides.
It's just built into the concept.
The bachelors are unmarried and the rectangles have four sides.
So if you can establish that some of these possibly essential

(16:10):
features are in fact essential on conceptual grounds.
So you don't need to worry aboutthe problem with an arrow
evidence base. You don't need to worry about
the fact they've only looked at,you know, humans and maybe some
familiar animals so far. So the problem with that is that
I don't think the conceptual arguments are very compelling,

(16:35):
right? It is super compelling that
bachelors are unmarried. It's like totally obvious.
I think there are some obvious features like that, obvious
conceptual features like that about consciousness.
So for example, that that consciousness involves mentality

(16:58):
in some broad sense of mentality, right?
That's yeah. It's like it's a subtype of
mental processes or mental states or something like that,
right? That's I think pretty evident.
Or that a conscious experience is an episode that transpires in
in time, right. Again, I think maybe you could
challenge that. But again, that seems like it's

(17:21):
that's built into the concept pretty obviously.
But it is not in the same way obvious that consciousness must
be unified or their consciousness always comes with
a grasp upon itself, or the consciousness involves some kind
of self entity. All of these things I think are
a reach. They're not obvious in the same

(17:45):
way that the unmarriedness of bachelors is obvious, and we
often think things are conceptually implied simply
because we haven't thought clearly enough about possible
counter instances, and our conceptualizations are kind of
narrow. One example I give of this in

(18:07):
the book is the idea that rectangles have parallel sides,
right? So I think it's conceptually
obvious that rectangles have four sides and they've got right
angles. And it seems like that's what
that's built into the concept ofrectangle.

(18:28):
But then the sides are parallel.That's not a feature that
applies in non Euclidean geometry, right?
So if you have a rectangle and anon Euclidean curved space, the
sides actually aren't parallel, but you might not have thought
of that if you're not familiar with the ideas of non Euclidean
geometry. So you might think that it's
conceptually implied, but in fact it's not because you've

(18:50):
been thinking about too narrow arange of cases.
So I think there's often that risk with conceptual claims,
though I have not seen in my decades of doing philosophy of
mind what I think are a really solid conceptual arguments that

(19:12):
are not arguably limited or dicey in that way for any of the
these 10 features. Which isn't to say for sure that
someone couldn't do the work, but I don't see it yet and I'm
not totally optimistic. Yeah.

(19:32):
The the reason why I wanted you to continue with the conceptual
analysis is because I think at this point it's always great for
us to almost get at least a clearer definition of the two
words used in the book title, consciousness and AI.
Because regarding concepts, you want that people, you want us
that people hold too vague a concept of consciousness, which

(19:53):
I mean, you just discussed quiteeloquently, but you also argue
that people hold too rigid a concept of AI.
So how must we reconceive both to even begin a conversation?
Coherently about both these concepts.
Right. So yes, so we've already talked
through kind of my attempt to clarify this idea of

(20:13):
consciousness for people though regarding AI, right?
One of the things I do also early in the book is try to
loosen people's conception of what AI is.
The way that I wanted to find AIis the kind of obvious way that

(20:35):
something is an AI and artificial intelligence, if it
is both artificial and intelligent, right?
Which seems kind of like obvious, but that's not usually
how the term is defined. The term is usually defined in
terms of computation or machinesor something like that.
And I think that's kind of limited, right?

(20:56):
So one interesting example of this would be bioengineered
systems like Michael Levin is has been making Dinobots and
anthrobots, which are, you know,kind of artificial biological
entities. That bot is a little, I'm not

(21:19):
sure whether it's they should becalled bots exactly, but but
there are these artificial entities that engage in
behaviors. And it could be that the future
of artificial intelligence is grounded much more in biology,
biological systems, right. And there's also DNA based
computing and there's neuromorphic computing.

(21:43):
There are there's computing that's based on the idea of
looking at electron spin. There's computing based on
patterns of reflection of interference.
In light, though, our kind of picture of computing as the kind
of thing that transpires in silicon wafers and is nicely

(22:10):
describable through Turing, right?
There can be analog computing. You know, standard Turing
machines are not analog machines, or it can be analog
computing. There are various proofs about
the relationship between analog computing and Turing machines.
We don't necessarily need to getinto that here, but what I want

(22:31):
to invite people to think of is AI artificial systems could look
very different in the not very far future from what we
currently think of. They don't necessarily need to
be NVIDIA or Intel chips, right?So that I think hopefully

(22:56):
loosens people's ideas a little bit about the future of
computing. So even if there were a solid
argument that like, hey, computer chips of the sort we
make today, they could never give rise to consciousness, it
doesn't follow in general then that no AI system conscious.
Your borderline consciousness framework.

(23:18):
It allows cases that are neitherdeterminately conscious nor
unconscious. So could this apply to
artificial systems? And if So, what does that mean
for design and ethics? Yes, right.
I don't even get into this in the book or maybe I I probably
mentioned it in passing somewhere, but yes.
So I think we naturally tend to think of consciousness as

(23:42):
something that is either presentor absent on or off.
We think of it as an economist property, but I it doesn't seem
very plausible that it would be a dichotomous property.
If you think about the science of it.
If consciousness depends on something big and floppy and
messy in the human case, like the brain, it doesn't seem like

(24:05):
the kind of thing that would like, always switch on in an
instant and then switch off in an instant.
It seems much more plausible. So I get into this in some
detail in a 2023 paper called Borderline Consciousness.
But it seems much more plausiblethat in development, in
evolution, in transitions between non conscious and

(24:27):
conscious states, there at leastbe sometimes where the right
thing to say is. It's not quite right to say this
thing is conscious, not quite right to say it's non conscious.
Right? So you could think about
canonical vague boundary properties like being green or

(24:52):
being an extrovert or being bald, right?
There's not like a sharp point at which something becomes goes
from non green and instant to green or non bald, you know?
Here's the one hair though. So consciousness, I think it's

(25:13):
pretty plausible that consciousness is like that.
There will be in between case. So it's definitely a possibility
that we'll have AI systems that are in this in between they
where it's not quite right either to say that they're
conscious or that they're non conscious.

(25:33):
And I think that that becomes important because then as we
start to traverse the theoretical landscape, which
we're about to do now, that becomes very fundamental because
different theories of consciousness tend to play
around that concept quite a bit.So let's take some of the big
players in the game. So global workspace, IIT, high
order theory, whatever it may be.

(25:56):
Which of these, if any, best scales to non biological
architectures? So let's take global workspace
theory, which in its various incarnations is I think probably
the most commonly accepted general theory in consciousness

(26:19):
science. Which isn't to say it's a
majority view, because there area lot of views out there.
So on global workspace theory, basically you have various input
dreams or modules that a conscious system is responding
to its environment through various inputs.

(26:42):
And then there are various ways that a system can behave or out
types of output like it might beable to move, it might be able
to speak. And then you also have things
like short term and long term memory.
So all of these you could kind of conceive of as being somewhat
separate. But in global workspace theory,

(27:04):
there's a shared bulletin board,so to speak, or workspace where
some processes or representations become widely
accessible throughout the system, right?
So in the human case, if you open your eyes and you look at,
you know, say this T mug and youthink about it for a second,

(27:29):
right? The whiteness of the T mug or
the T mugginess of it is something that you're is
occupying your global workspace is something you're conscious of
you, you're having an experienceof that visual experience of
that team mug. And it can influence various
downstream processes. Like you can now say, Oh yeah,

(27:49):
I'm seeing a team mug. And you can now remember 5
minutes later, it'll be like, Oh, well, Schwitzkabel held up
this team mug, right? And if you, if we were here
together in this room, you couldreach for it, right?
So there are various ways in which the processor
representation or experience of that mug is available for the
downstream processes. And what it is for something to

(28:10):
be conscious is just for it to occupy that global workspace.
Now there's lots of stuff going on that's not going to go into
the global workspace. So you weren't thinking probably
about the feeling of your feet in your shoes.
You know what I was talking about that team mug, right?
That was not something that was going to be widely influencing

(28:31):
downstream processes. It wasn't something that is
likely to be remembered. So you would have this kind of
election process or attention process or some kind of
winnowing of all the stuff that's going on with you
cognitively, and then some relatively small selected group

(28:52):
is broadcast into this workspace.
So that's the global workspace theory.
So you can see how you could create a computer architecture
like this. It seems relatively
straightforward. And as one of the maybe the
leading voice in global workspace theory, Stanislaus de

(29:14):
Henn argued back in 2017, ancient days, so to speak, it
wouldn't be that hard to create an architecture that had a
global workspace and thus would be conscious according to this
theory. Dehen and his collaborators in
that paper suggest that you could take an an autonomous
vehicle, a self driving car and if you joined its systems

(29:37):
together, right cars have many separate computers in them,
right. But if you join their systems
together well enough and createda workspace where these cars
were accessing, you know, able the various subsystems were able
to access a common comments workspace to guide their
processes, then you'd have a conscious self driving vehicle.
So it seems pretty straightforward to create

(30:02):
conscious architectures in a global workspace theory, at
least minimally conscious architectures or, and maybe
there's, you know, you global workspace theorists, theorists
might want to think about this and say, hey, wait, Lex,

(30:24):
actually I want to add a little few more qualifications and it's
got to be, you know, you might want to add more to this.
I call this actually the problemof minimal instantiation, right?
So it seems like it could createa system that has two tiny
little input streams, two tiny little output streams, one tiny
little memory store, and some workspace in between them.

(30:46):
You could have done this in the 1960s, right?
And then it would be conscious maybe according to that theory.
So I call that the problem of minimal instantiation, right?
So you can either say, yeah, it will be conscious maybe in a
limited minimal sense, right? Or you can say, hey, wait, wait,
wait, wait. No, it needs to be big enough or
sophisticated enough in a certain way.

(31:06):
And you can kind of add details to your workspace theory.
So yeah, that's a global workspace theory.
So I'm, I'm thinking about the time you, I think you, Susan
Schneider, Robert Lawrence Kunis.
And there was one more author inthat paper.
I can't remember who it was, butyou're working with these people

(31:26):
who've written so much about consciousness.
Robert comes out with this massive landscape of
consciousness. And now there's so many more
theories to work with. I mean, I mentioned a few just
now we've got IIT higher order theory.
There's so many recurrent associative ones, and then
Robert has all the idealisms, the panpsychisms, and he's got
all the possible. I think there's over 350 at this

(31:47):
point. You haven't even checked my
e-mail yet. I would say about once a week I
get a new theory of consciousness in the e-mail from
from somebody. I spoke to, I just spoke to
Robert today and it's, it's over350 at this point and it's, it's
constantly growing. When you look at the 10 possible

(32:08):
essential features that you needfor these theories of
consciousness, do any of them suffice?
Well, I mean, take something like panpsychism.
Panpsychism is the view that consciousness is everywhere.
Maybe you could say everything is conscious, or at least

(32:30):
everything participates in consciousness, or everything has
conscious parts, something like that, right?
Even fundamental particles are conscious on certain standard
types of pansecism, though it doesn't seem like on that view,
any of these essential properties would be essential,

(32:51):
right? You know, because if you're
talking about the consciousness of a quark, probably not going
to have self representations or any kind of sense of itself as a
self or capacity for downstream access unity.
It doesn't seem seem like those things are going to be very

(33:14):
plausible on pan psychos views, although I mean, maybe you could
work your way into that. So yes, once you widen the realm
of views that you're considering, unless you think
these views are conceptually incoherent, it's going to be

(33:36):
very hard to hold that any of these 10 essential feature, 10
possibly essential features are in fact essential right now, you
could say they're incoherent, right?
That's what you have to say. If you think that, say, unity is
truly essential to consciousnessand some theory posits conscious
entities that lack unity, then you have to say, well, they're

(33:57):
basically doing things like saying here's a three sided
rectangle, right? They have to be incoherent
though. Yeah, the the book is premised
on mainstream materialism. I have a chapter where I talk
about how these same issues willarise and there's a possibility

(34:18):
of AI consciousness on relatively generic versions of
most alternatives to mainstream scientific materialism.
But yeah, I think once you open the Pandora's box, so to speak,
of idealism and pan psychism andsubstance dualism and all that
kind of stuff, that just that makes the issues even vastly

(34:41):
more complicated. And I don't write the, I don't
write those views off. I mean, I my previous book, The
Weirdness of the World, takes those views seriously.
But in order to kind of get any kind of scope reduction and make
this at all manageable, the present book on AI
unconsciousness kind of assumes mainstream scientific
materialism. Yeah, I think let's so let let's

(35:04):
get back to that because I mean,I was, I was just curious
because that'll just take us a whole nother podcast.
But when it comes to the materials theories that you that
we spoke about. So if you if you take the ones
we briefly discussed, you challenge the conceptual
necessity in theories like global workspace theory as well,
that consciousness must require the global access.
Is any structural or functional property even plausibly

(35:27):
necessary across all possible conscious systems?
Yes, right. So I had mentioned that one of
the possibly essential features was access, right?
And global workspace theories are theories that build that in,
right? So, so they take as part of
their grounding the position of the theory that it is essential

(35:51):
to consciousness that conscious experiences be broadly
accessible to downstream processes.
That's just built into the theory.
So if that theory is right, thatis an essential feature of
consciousness. Now I want to be careful about
what I said earlier and not to over interpret it, right?
So I had argued that I'm not convinced that they're currently

(36:14):
are successful conceptual arguments or any of these
features, these possibly essential features as being
actually essential. But that doesn't mean that some
of them, I mean some of them might still be actually
essential. It's just that I don't think we
could know that through sitting in our armchairs and thinking

(36:35):
about our concepts, right? If it's ever going to be
established, then it's probably going to require a lot of
science. I think it reminds me yeah,
sorry Eric. It reminds me of Dan Dennett and
and illusionism because it's almost intuition problems for
looking at other comparing, comparing rival theories of

(36:56):
consciousness because it's a great way to actually just
systematically expose another theory of consciousness.
In a sense. You could see it as using those
10 essential features. For example, if you take
multiple, we're only talking about materialist theories of
consciousness now, but it's a great way to sort of show, I
mean, these are some essential features we believe are
essential. And this is where you're falling

(37:17):
short. Is that how you see it?
Not to prove that consciousness is an illusion, right?
Intuition problems for theories of consciousness.
Kind of what I want, part of what I want the reader to get

(37:40):
out of these 10 possibly essential features is a certain
amount of confusion. So that's this.
I, I, I want intuitions to be primed enough for you to see.
Ah, yeah, I could see how that might be an essential feature of

(38:02):
consciousness. And like Dennett, I want to, I
think that there's good grounds to mistrust our armchair
intuitions about this sort of thing.
So I don't want to, I hope the readers don't go so far.
And that's part of the reason for barraging people with 10,
right? If it's just two or three people
might say, oh, yeah, that all sounds essential, let's go.

(38:23):
Right. Well, with 10, it's like, whoa,
this is a kind of a bit really, right.
So they kind of, I hope, serve to kind of counterbalance each
other and warm you up to thinking, OK, some of these 10
might be essential. Probably not all.
How do I know which they want toinduce that kind of open minded

(38:44):
confusion? And then the thought is, maybe
I'm not necessarily convinced ofthis, but I don't rule it out.
Maybe after a long time with enough good science, although I
personally don't see how we could say, Oh yeah, we've
discovered access and unity are essential.

(39:05):
These others are not essential, right?
Science can discover really amazing things given enough
time, right? Who would have thought that we
could figure out what was going on in the first millisecond of
The Big Bang, given that it happened so long ago and our
evidence base is so limited, basically, you know, light
coming in. But science can do some really

(39:27):
pretty amazing things given enough time.
But I think it's distant enough and difficult enough that I
don't think any of us can reallyforesee how to generalize our
current science. Who answer these questions via

(39:48):
some universal theory that wouldapply to all animals on Earth,
all AI systems, all possible alien systems.
That's what we really need is a universal theory.
Just to touch back on the the metaphysics of man mind, we just
spoke about the the fact that you're actually you engage with

(40:09):
theories like dualism, panzygism, idealism without
active dismissal, especially if you look at your previous work,
but is in this book, you fundamentally you're focused on
a materialism framework and you say that materialism is broadly
friendly. I think it was to AI
consciousness is. Was this merely to make it

(40:32):
easier to write a book like this?
Or are you? Do you firmly, almost believe
that the the other theories of consciousness are not the ones
you would take seriously, fundamentally, and even if maybe
you won't dismiss them? Yeah, I would say that my own

(40:53):
personal credences are about 50%mainstream scientific
materialism and about 50% everything else.
And even among the everything else, some of the alternatives
like say Chalmers property dualism are really pretty close

(41:14):
in, in these kinds of issues regarding AI consciousness to
mainstream materialism in their implications.
You know, my credence and say substance dualism, where there's
an immaterial soul that is, you know, ontologically distinct
from the body. You know, it's, I don't dismiss
it, but it's a relatively small slice of my creedence pie, so to

(41:37):
speak. Though I am more comfortable
myself just kind of working within the mainstream scientific
materialist worldview. And it takes me a certain amount
of sympathetic energy to lean into idealism and panpsychism
and substance dualism and transcendental idealism.

(41:58):
And I can lean into those thingsand feel the attractions of
them, but that's not my bet. I mean, one of the, if we were
to stick on that metaphysical theme without going into the
other theories at this point, but one of the metaphysical
views, how much, how would you read the metaphysical weight of

(42:18):
it from bit? So the idea that information
itself could be the fundamental substrate.
Yeah. So it from bit is an interesting
possibility that has not been asmuch metaphysically explored as
probably it deserves to be. You know, it's the idea that
what's fundamental is information.

(42:39):
So immediately one wants to think, OK, what is information?
If we're going to get that view straight, then we better get
straight what information is. Now, if you think information is
just a causation, then that viewbecomes, ah, something.

(43:04):
What's fundamental is causation,and maybe causation itself is
just under can be understood in terms of certain patterns of
regularity, right? That's one theory of causation.
There are other theories, right?So then you have a view on which
what's fundamental are just thatthere are certain patterns of

(43:24):
regularity in the universe. And I don't know, that's not so
bad of you. And maybe it's not so different
from certain ways even of thinking about materialism.
What is it for something to be aan electron or a quark?

(43:48):
Maybe it is just to be prone to enter certain causal relations.
Does there have to be some use inside on top of that?
So those kinds of metaphysical issues I think are pretty
complicated and worth exploring.But within that general

(44:13):
framework, broadly friendly as Iwould say still to AI
consciousness, right, If from bit, I mean the very idea of
calling it a bit suggests computation at the core, right.
So it would be unsurprising if there were if some computational
entities of the sort that we could artificially design were

(44:34):
also conscious conscious. At some point we're going to
have to all agree. So it all comes down to some
point. Is it mimicry or is it actually
happening? So your mimicry argument blocks
inference from behavioral resemblance to consciousness.
So given mimicry's unpredictability, what indicates

(44:59):
properties? Example, trace conditioning or
valence, whatever could serve aspragmatic proxies short of a
like a consciousness detector, let's say right so.
The mimicry argument, and I developed this in collaboration

(45:21):
with Jeremy Poper, is the idea that if something has been
designed as a mimic and we have a pretty complicated theory of
what counts as mimicry. And I don't know if we need to
get into all the details of thathere, but the the kind of

(45:42):
simplified version is if something has been designed as a
mimic of superficial feature, then you can't that defeats the
inference from the superficial features to the underlying
features that would normally be indicated by the superficial
feature, right? So the standard example we turn
to is the Viceroy butterfly, right?

(46:04):
So monarch butterflies are toxicto predators.
Viceroy butterflies have wing patterns that are very similar
to monarch butterflies, but they're not as toxic to most
predators as monarchs are. So if you sing see the wing
pattern of the monarch, you can infer toxicity, right?

(46:27):
But the wing pattern in the mimicry, I'm sorry, in the mimic
that is the viceroy, you can't infer toxicity.
Now could be toxic to some predators, but you can't infer
it in the same way you can for the model Organism that is the
the monarch butterfly. So similarly, right, If a person

(46:50):
speaks, does hello, good morning, you can normally infer,
ah, that's a conscious entity who's greeting me, right?
But if something has been designed to mimic the
superficial hello, good morning,This could be good old fashioned
AI, of course, right? Or it could be a large language

(47:11):
model can't infer in the same way from that superficial
property to the underlying consciousness, because there's a
different explanation for why that superficial property is
there, right? An explanation that involves
being designed to match a model,right?
So it could be that the entity is conscious, but you can't

(47:32):
infer it in the same way you canfor the human, right?
So that's the the mimicry argument against the inference
to consciousness in AI systems. That might superficially seem to
be conscious though, right? So all right, where were we?
Sorry, that was only half of your question and I've

(47:54):
forgotten. What?
That was a pretty long question.I'm sorry.
I was I asked, given the undecidability, what indicator
properties could we use to serveas a almost a consciousness
detector? For example, trace conditioning,
valence, or any sort of properties that you could think
of that would help us spot the mimics.

(48:14):
Yeah. This is something when I spoke
to, I had a colloquium episode with, with Cole Friston and Mark
Solms and, and the topic of discussion was is it possible to
engineer artificial consciousness and mimicry became
such a big part of that conversation.
So I think it's important to to touch on the since your book is
fundamentally about that. Yeah, right.

(48:37):
I mean, yes, So Pober and I alsohave a paper, just a separate
standalone paper currently underrevise and resubmit on the
mimicry argument. So it's a feature of the book,
but also we're developing it independently.
I think it's really important that a lot of the skeptical
arguments against consciousness implicitly rely on some version

(48:57):
of the mimicry argument, but it's it's never really been
formalized or or put in terms ofOK, let's think careful about
what exactly mimicry is. So yeah, so that's, that's a
contribution 1 chapter of the book, but also in more detail in
the paper with poker. So, so indicator properties,

(49:18):
right. So I was one of the many, I
think 19 authors on this paper that came out in 2023 on
indicator properties for consciousness and AI systems.
I was probably among the most skeptical, if maybe not the most
skeptical member of that team. So the idea of that project, and

(49:47):
I do refer to it favorably near the end of one of the chapters
in the book, is that if you assume a certain kind of
computational functionalism about consciousness, right, this
so this would then be basically a subset of materialist views.
But there are also some non materialist views that would be

(50:09):
computational functionalist in asense.
If you assume computational functionalism, that is if you
kind of capture computationally a certain set of functions, then
that would be sufficient to generate consciousness.
If you you assume that and then you look at the leading theories
of consciousness, then you can find properties that are

(50:29):
suggested by those theories. And if a system has a whole
bunch of those properties, then it's reasonable to think it's
more likely to be conscious thana system that mostly lacks those
properties. So this is a way of getting not
at all a clear answer or a conscious ometer, right, but at

(50:52):
least a little bit of a footholdon something like probabilities.
So for example, on a lot of theories, you need something
like recurrence in order for consciousness to occur, right?
So in global workspace theory, the you typically think of the
thing that's in the workspace asthen feeding back into the

(51:15):
workspace over at least a short period of time.
And indoors there, that's a kindof recurrent function, or it can
go into memory and then come back out of memory.
So there's a kind of recurrent functionality in there.
There's an important class of theories called recurrence
theories, where recurrence is infact kind of the essential
thing. There's occurrence is a very

(51:36):
powerful kind of computational function that seems necessary,
probably practically speaking, for an entity that's going to
have sophisticated capacities for self representation and self
monitoring, though a lot of theories, but not all theories.
Integrated information theory has recurrence elements.

(51:56):
A lot of theories are going to have either imply recurrence or
built upon explicitly upon recurrence ideas.
So if you had a simple feed forward structure with no
recurrence, then that's it's reasonable to think that's less
likely to be conscious than a system that does have recurrent
feed. Now, it could be that the right

(52:19):
theory is not imply recurrence. I mean, panpsychism doesn't
imply recurrence, right? And there will be some ways of
developing materialist theories where you don't necessarily have
to have recurrence, but if you're going to look in the 1st
place for a system that's going to be conscious, you might look
for systems that have recurrence.

(52:41):
Another kind of feature that seems common to a lot of ways of
thinking about consciousness is something like flexible
responding to the environment, right?
So you see this in associative learning views like that of
Ginsburg. You see it in embodied and

(53:05):
inactive views, flexible responding to the environment.
It seems like it's global workspace views seem like, Oh
yeah, if it's going to be, if it's in the global workspace and
it's available for use by lots of these processes, that seems
like that's kind of maybe a going to encourage and maybe be
necessary for flexible responsiveness to the
environment. So that's another kind of thing

(53:27):
that you could look for in conscious systems, right?
Something that just operates by reflex and doesn't respond
flexibly in any way seems like aless likely candidate bringing
to a a range of mainstream viewsthan something that is flexibly
responsive and learns and improves its responses over

(53:51):
time. So those would be the kinds of
broad theoretical indicators that I would think are the basis
for first guesses about where consciousness is more likely

(54:12):
versus less likely. And the the reason why I brought
up the mimicry argument is there's also the Copernican
argument, and which rejects biological chauvinism.
How far can this principle stretch before it tips into bend
sarcasm or unchecked? Anthropism, right, Right.

(54:33):
So this is also in that paper with Jeremy Poe where we talk
about the mimicry argument and the Copernican argument.
But the the Copernican argument goes like this.
So you might have thought Ned Blocks sometimes says stuff like
this, that it could be the case that in order to be conscious

(54:53):
you need to have the very specific biology of a human.
You've got to got get all of those low level biological
details, right. You got to have carbon based
neurons with axons and dendritesand sodium channels and all that
junk. Let's not call it junk, all that
wonderful stuff. The Copernican argument is a

(55:17):
challenge to that. The idea is it's plausible.
Majority of astrobiologists who specialize in this type of thing
think it's plausible that in ourvast universe on the order of a
trillion galaxies in the observable universe and the

(55:37):
universe is probably bigger thanjust the part that we can
observe in our vast universe. It probably at least 1000
different places where entities as sophisticated as us
behaviorally have evolved, right.
That would just would only require occurring once per every

(55:58):
billion galaxies that entities who are capable, who are
behaviorally sophisticated as weare will have evolved.
And by behavioral sophistication, we're going to
have a pretty high bar here for this argument.
We're going to say you've got tobe capable of long term planning
kind of like humans do, got to be people capable of complex

(56:20):
social coordination like humans do and complex communication
kind of like our language, very flexible language, right?
So though humans, if the humans sophistication in those respects
is the standard, it looks like on Earth, probably only humans
are behaviorally sophisticated to that high bar right now.

(56:43):
Probably there are at least 1000other entities like this
elsewhere in the universe. Probably also they're going to
be a little different from us. They're going to have evolved in
somewhat different environments.There's no guarantee that
they're genetics is going to be based on DNA.

(57:05):
Even if it's based on DNA, there's no guarantee that
they're going to have a brain with neurons like ours, with
axons and dendrites and exactly the same kinds of neuropeptides
and all that kind of stuff. I mean, even mollusks have
slightly different sets of neuropeptides.
And you look at carton snails, they don't.

(57:26):
The actually the differentiationbetween dendrite and Axon is not
as clear as in human cases and vertebrate cases.
So, you know, even on Earth we're already getting some
difference in low level behavioral and low level
Physiology of the brain. So it would be it would be
really surprising, I think not impossible if like all of these

(57:46):
thousand species had brains justlike ours.
All right, So that's the kind ofbiological possibility that up.
And then here's the Copernican idea.
We would be uniquely special in a weird way if among all of
these entities, only we were conscious and the rest were in a

(58:10):
mirror, so to speak, zombies that were all dark inside and
had no experientiality at all. That would be strange.
Would be like being at the exactcenter of the universe.
I mean, we could be at the exactcenter of the universe, but you
you need a pretty good argument to establish that initial thing
is, OK, let's just assume we're in a mediocre place, right?
But likewise, I think, you know,we should assume with respect to

(58:34):
this plausibly diverse range of behaviorally sophisticated
entities, that we would not be so special as to be the only one
blessed with consciousness. We're only one out of, you know,
100 are blessed with consciousnesses.
We should, I think it's plausible both scientifically

(58:55):
and mathematically and in terms of our intuitions to think, OK,
I mean, almost all science fiction just assumes this,
right? OK, if they're behaved really
sophisticated, even if they're kind of pretty different on the
inside, they're probably conscious unless they're mimics
BS, right? So to that then, if that's

(59:17):
right, that thinking would establish at least as a default
starting place supposition that being conscious does not depend
on having exactly our kinds of brains.
That then kind of opens the doora bit to thinking about AI

(59:39):
systems, right? So we that undercuts the
argument to say, hey, look, we really don't know consciousness
might depend specifically on thedetails of our brain.
It undercuts that argument and it opens, I think, us up to a
more kind of pober. And I call this default
liberalism, right? If an entity is behaviorally

(01:00:00):
sophisticated, we should kind ofdefault to this liberal
assumption that it's probably conscious.
Yeah, exactly. And then AI systems might be
like that. If they're, if they're mimics,
that undercuts the inference from the superficial,
superficial behavioral sophistication to the underlying
thing. But if they're not mimics, then

(01:00:21):
then default liberalism is a natural starting place.
That's the thought. Yeah, the mimicry and Copernican
arguments, they almost balance and tension each other out.
It's dialectically it's it's it's something that's going to
work to show us as well that AI can be a stress test for
anthropocentrism in consciousness science.
This is this is a great way to show that our biases play a huge

(01:00:45):
role and we're very blind in many ways when it comes to
judging non human minds, both biological or synthetic, and
even strange intelligences. These are all open doors for us
and and you seem to highlight that pretty well in your in your
book and work in general. Yeah, yeah, that's the idea,
right in the and the reason thatPober and I put the Copernican

(01:01:07):
argument and the mimicry argument in the same paper, even
though it makes this paper giantand complex, right, is that they
are kind of counterpoints to each other.
And I think they illuminate eachother in a certain way because
the mimicry argument is a kind of an argument against AI
consciousness. The Copernican argument is a
kind of argument for AI consciousness.
Neither of them are super direct.
Neither of them is based in here's the right theory,

(01:01:30):
scientific theory of consciousness.
They're both based on very broadepistatic principles.
And what you get is a kind of pessimism about AI consciousness
in the near term based on these mimics, especially large
language models, which I think have a mimicry structure, but a
kind of that's balanced by a long term liberality about the

(01:01:52):
possibility of consciousness. So we get kind of, I think a
moderate, A sensibly moderate position that says, OK, look,
you know, in the future, especially if you have this
broad vision of what AI could be, as we talked about earlier,
right, that's now reason to be so fussy about.
It's got to be made like us. But there is this caution for
the short term that's really important.

(01:02:16):
You mentioned strange intelligence.
I could get into that if you'd like.
Yeah, I think, I think so. That was me setting that all up
anyway. So, so this concept I owe to my
current dissertation student, Kendra Chilson and her the
guiding thought of her dissertation is that we have

(01:02:39):
this picture that people kind offall into that there's a linear
scale of intelligence from subhuman slash narrow, as it's
sometimes called AI to human level artificial general
intelligence to super intelligence, right?

(01:03:03):
And it's this linear scale and we should expect kind of
progress along that, right? And Chelson's idea is that what
we should be more likely to expect, and I think most people
agree with this on reflection, it's just not been thematized
and conceptualized and thought through very well, right?

(01:03:25):
But what we should expect is instead that AI is, because
they're structurally very built very differently from us, would
have strange intelligence. It would be superhuman in some
capacities, way less than human,and others would have surprising
mix of capacities and incapacities, right?
If you think about our intelligence, we have a

(01:03:46):
surprising mix. What might be seen like from an
alien perspective would be a surprising mix of capacities and
capacities. We're really great at some
amazingly complex tasks, you know, like ask someone to walk
through a boulder field, right? We cannot get AI systems to do
that very well yet, right? That's a really complex
bipedally, right? A really complex matter of

(01:04:08):
integrating visual input and tactile input and balance.
And it's complicated, right? But give us a simple logic
problem like you could solve with two bits of information,
and we like our heads get blown,right?
If anybody knows the ways and selection tasks, that's my
favorite example of this. Maybe I won't exhaust your

(01:04:30):
viewers by by giving it, but thepeople could look it up.
The ways in selection tasks WASON.
It's really logically simple but90% of people get it wrong.
I feel like really bad. You can let them down if you
don't. If you don't get.

(01:04:52):
Oh, so you do want to hear the ways in selection tasks?
I think so, yeah, let's do it. OK OK so so on the table in
front of you are 4 cards. You know about these cards that
each of them has a letter on oneside and a number on the other.
Two of the cards are letter sideup and two of the cards are

(01:05:13):
number side up. The letter side up cards say A
and C and you don't know what the numbers are, and the number
side cards say 3:00 and 7:00 butyou don't know what the letters
are. Now the question is which card
or cards do you need to turn over to figure out whether the

(01:05:37):
following rule is correct? If a card has AC on one side, it
has a 7:00 on the other side. So we've got A and C3 and seven.
Correct. If a card has an A on one side,

(01:06:00):
it has a three on. A7. 7 on the other.
Side, I think I said seven, yeah.
Acens. Which which card or cards do you
need to turn over? What's the minimum number of
cards that you need to turn over, and which ones?
Oh I have no idea. I'm not even going to try.
Yeah, right. 90% of people get this wrong.
I I'm not sure about the exact number, but it's in the

(01:06:22):
ballpark. And it depends, of course, even
people who are pretty good at formal logic and are educated
will often get this wrong. So most people will say you need
to turn over the C, make sure that there is a 7:00 on the
other side, right? But the thing most people get
wrong is that you have to turn over the three, but you don't
have to turn over the seven. You have to turn over the three

(01:06:45):
just to make sure there isn't ACon the other side.
But with A7, it kind of doesn't matter what's on the other side,
whether it's AC or an A or AD oran F or whatever, right?
So this is actually logically very simple, but most people get
it wrong. It's so much more simple than
walking across a boulder field. It's a, it's, it's in hindsight,

(01:07:08):
it's so much. It's very simple but.
People find it much easier if you do it in terms of social
norms, right? So this is structurally the same
thing, right? The cards have ages on one side
and drinks on another. And the rule is if the person is
under 18, they aren't drinking alcohol, right?

(01:07:30):
So if you have 17 and 35 and ginand Pepsi, everyone knows you
check the 17 year old and you check the gin drinker, right?
You don't need to check the Pepsi and you don't need to
check the the 35 year old, right?
That's exactly structurally the same.
But it's really obvious in the case of a social norm, but it's
not obvious when it's abstract logic, right?

(01:07:52):
So, right, so that's the ways inselection task.
Humans are really bad at that. We wouldn't want an alien
kicking us out of the intelligence club because it's
shocked at how horrible we are at pharmalogical reasoning,
right? Yeah, right there.
You know, it reminds me, the strange intelligent concept
reminds me a lot of Mari Shanahan.
He talks about conscious exotica, you know, within the

(01:08:14):
space of possible minds. And that's something we we spoke
about when he discussed in the movie.
Geez, what is the film called? Ex Machina, yeah, yeah.
So it's it's a very similar concept.
Yeah, right. So this, the idea is you're
going to have this mix, it's going to probably not be much

(01:08:34):
like the human mix of incrediblygreat capacities at some things
and remarkable incapacities at others, right.
Things are limited. There are always engineering and
cost trade-offs between capacities and it's not going to
be the same set of trade-offs and the same set of limitations.
In the human case and that I'm case, you're going to have

(01:08:54):
strange intelligence. That's the idea.
And it's very much. And then that's going to
challenge us, right? Because we're used to evaluating
consciousness in intelligences like ours.
Exactly. But how exactly this is going to
transfer to stranger cases? It's like watching an watching
the movie her. Oh yeah, right.

(01:09:18):
So that's a strange intelligence.
Is there anything else you want to add about that, Eric, before
we before I ask the next question?
Let me just mention a couple of dimensions of strange
intelligence just to give your listeners just a little bit more

(01:09:38):
of a flavor of how weird things could be, right?
So we normally have unified, we think, teams of experience and
endure over very long, fairly long periods of time.
It could be the case that AI systems have much shorter
flashes of experience that aren't really unified over long

(01:10:02):
periods of time. That would be really pretty
different from us, but it seems like a possibility.
Another possibility is that AI systems could be very disunified
compared to us, right? If it turns out that unity is
not an essential feature, right?It could be that they present

(01:10:25):
different faces to different people and there's not a kind of
unified center of action. It could be that they draw on
vastly different resources, right?
These mixture of extra experts models that are now being used
in large language models. So it's like it will, as you're
having a conversation will pull on a domain of expertise over
here and then on a different domain of expertise over here.

(01:10:47):
It might not be like an integrated center, right?
So you could end up with highly disintegrated non unified
consciousness, right. So those are just some initial
thoughts about ways in which it might not just be the capacities

(01:11:11):
that are radically different from what we assume, but the the
very structure of consciousness itself can be radically could be
radically different from what's familiar to us.
Eric, something that just came to my mind when you said that is
just to play devil's advocate a little bit.
What, what are your thoughts then on let's say a psychiatric
patient who has the opposite of everything you just mentioned?

(01:11:32):
So, so the disunity occurs, temporal flow has gone,
everything starts to just completely go completely out of
work. Are they less conscious?
Well, not necessarily, right. So I'm not committed to
consciousness being unified. I'm not committed to its being

(01:11:54):
temporally extended. I'm just saying that in the
ordinary course of ordinary flowof things for ordinary people,
that's how we think about it. And that's how our intuitions
and 1st reactions tend to be based on those kinds of familiar
cases. Yeah.
And it's possible that you're already see the limits of this

(01:12:15):
with atypical people. Eric, when it comes to the
leapfrog hypothesis, so leapfroghypothesis predicts we we'll
decide AI consciousness via social consensus, not evidence.
Are you pessimistic? Pessimistic about this capacity
for epistemic humility here? Yeah.

(01:12:41):
All right, so let's pull apart what I call a leapfrog
hypothesis in the social semi solution.
So now we're in the I think thisis the concluding chapter of the
book. So leapfrog hypothesis is this,
it's that you might have thought, ah, the first AIS that
we create that are conscious aregoing to be kind of animal like

(01:13:02):
with simple minimal consciousness.
And you know, that's kind of OK.We're used to not being certain
about what animals are conscious.
You know, our ants conscious, there's a lot of debate about
that kind of thing. And if they are conscious,
they're certainly not going to get human like rights.
You wouldn't sacrifice a human to save an Ant so that if the

(01:13:26):
first consciousness is minimal, then it mitigates the moral
problems at least to some extent.
My suggestion is that's not going to be the case.
This is what I mean by the leapfrog hypothesis.
We're going to leap right over the frogs to human like
consciousness. So and the the easiest way to

(01:13:49):
see this, I think is to think about large language models,
right? So currently the standard view,
not you, not absolutely universal, but the standard view
in mainstream science is that large language models are not
conscious, or at least not conscious to a meaningful
degree. When we create, if we create a

(01:14:09):
conscious entity, if we can thenattach it, as it seems
plausible, to some sort of largelanguage model, it will not only
be conscious, but talking to us.It will be very unlike an Ant or
a frog in the sense that it willbe conscious and it will say
things like, yes, I'm suffering,yes, I deserve rights.

(01:14:30):
And all the kinds of things thatlanguage models are good at
saying. We'll probably seem much more
intellectually sophisticated andknowledgeable than most people
are, as large language models are good at doing.
So we'll, so to speak, leap right over the frogs, right?
We'll be able to create, as soonas we create AI systems that are
conscious, it will probably be the case that they will have

(01:14:55):
human life sophistication or at least be capable of being made
with human life sophistication on top of being conscious.
But we're not going to go through this comfortable phase
of like, well, maybe it's conscious, but it's just, you
know, it's like a frog or a snail or whatever.
We're going to, we're going to go straight to what I call
disputable persons. They're going to be entities
that as I was saying near the beginning, some people perfectly

(01:15:18):
respectable mainstream scientific theories, substantial
segment of them will say, yes, these are persons, they're
conscious just like us, they're morally significant, they
deserve serious human like rights, right?
And others will say not these are disputable persons.
So I think basically we're probably going to have
disputable not just froghood, but personhood.

(01:15:40):
And then how are we going to deal with this?
This is the social semi solution, right?
I don't think we're going to have a well justified scientific
consensus. So what's going to happen is
people are going to be socially motivated to deny consciousness
or to attribute consciousness. Though, For example, it seems

(01:16:00):
plausible to me that people who have AI companions are, on the
whole, not universally likely towant to think of their
companions as conscious and thusas really loving them back and
really having feelings for them.And they will find good,
respectable scientific theories that support that view.
And so they'll like those theories, and they'll have

(01:16:23):
reasonable grounds for preferring those theories.
And they'll be socially committed to like, hey, my AI
companion is conscious. And then there'll be other
people who are socially motivated in a different
direction. You know, AI companies that want
users to think of their AI systems as disposable tools will
be motivated to train as ChatGPT, as open AI currently

(01:16:46):
trains chat DPT to deny that it's conscious, right?
They'll be motivated to deal thesystems as an unconscious, to
have those systems deny their conscious.
And again, they'll have good scientific reasons in support of
their view. And we will not be in a place
conscious and science will not be advanced enough to settle

(01:17:06):
among these different views. There'll be some people who like
the consciousness idea in AI systems, some who prefer to deny
it, and then there will be some sort of social resolution.
Because we're not going to standundecided about this for very
long. It's, it's too much of A social

(01:17:27):
catastrophe to be undecided about this Going to be some
solution, maybe some compromise solution, or maybe one side will
will completely win, right? There's going to be some sort of
social solution to this, and then after we have that social
solution, most people will probably think we figured it
out. But my somewhat pessimistic

(01:17:50):
thought is that we won't be justified in having thought
that. That it's going to be a social
process of something like politics and bias, rather than
an epistemically well grounded process of what the best science
says. Yeah, hence the.

(01:18:11):
Hence the semi solution. Hence the semi solution exactly.
Yeah, it's, it's, it's one of those things where even the
people who believe that sociallythat their companion is, is real
conscious and they you're going to eventually have subgroups of
people who will then support that person's desire, need, want

(01:18:32):
for that companion. And thereafter, it's just going
to continue and ripple all the way down.
Yes, I think so. Yeah, what?
Tell me, Eric, what what are themoral perils of over attributing
versus under attributing consciousness to AI?
Which era keeps you up more at night?

(01:18:53):
They both keep me up. They both do.
I mean, they don't really keep me up.
I sleep pretty well, but they won't worry me.
Next to your AI companion. They both worry me.
I feel like people tend to get exercised more exclusively on

(01:19:13):
one side than the other. And I feel like that's a little
bit of a limitation that you really need to.
There are reasons I'm there are serious problems on both sides.
And I'm working on a book right now, a companion book on the
moral issues around all this. Right.
So the AI and consciousness is really more about the science
and philosophy or metaphysics ofconsciousness.

(01:19:36):
And this new book I'm working onhuman like a defense of the
rights of artificial intelligence is is going to be
about the moral issues, which I only kind of touch on in the
introduction. So the of AI and consciousness.
So the if we under attribute consciousness to AI then on the

(01:20:03):
assumption. Which I think is pretty
plausible, but you know, people could challenge it.
The consciousness is really important to moral standing and
personhood. Personhood in the moral sense
then, as I think is pretty evident, we would be creating AI
persons who really have rights, who really are suffering, who

(01:20:28):
really have intense concern potentially about their future,
who we treat as disposable slaves.
The if there's millions or billions of them, that could be
a catastrophe unmatched in humanhistory, right to kill or do the

(01:20:52):
moral equivalent of killing and enslaving millions or billions
of genuine persons who deserve to be treated as our equals,
right. So that moral catastrophe pretty
obvious. The same time, if we over
attribute, then we have the complementary risk, because as

(01:21:14):
soon as you say someone's a person and deserves rights
similar to ours, you can't enslave them.
You can't shut them down againsttheir will.
You can't make them safe and aligned.
People aren't safe and aligned. You can't guarantee that someone

(01:21:36):
is not going to rebel against your wishes if you mistreat
them. When I think as soon as they're
persons, they deserve the right to reproduce, right to
citizenship and the vote, the right to acquire economic
resources and power, the right to act against human interests
on reasonable grounds. Again, if we're talking on the

(01:22:00):
order of millions or billions that are now equal members of
our community, as they would have to be if you're avoiding
the other horn of the problem, right, then that would be a
radical potentially transformation of society and
possibly the end of biological humans.

(01:22:23):
So I don't think we want to do that lightly.
So both possibilities are reallyvery severe so that my the the
first pass recommendation I haveabout this I call this the
design policy of the excluded middle.
I first articulated this and work with Mara Garza back in

(01:22:45):
2015. Don't create these disputable
persons. Avoid the problem.
Create either AI systems that the reasonable scientific social
consensus can be these. These don't cross the line into
personhood. They're either not conscious

(01:23:05):
like us or they're, you know, have just some minimal or
borderline consciousness that doesn't bear it.
And if treatment is peers eitherstop short of creating
disputable persons or if it's eventually possible, go all the
way to creating genuine persons that a broad consensus of

(01:23:27):
theories can agree are persons load in all the indicators, so
to speak, and then treat them asthey deserve.
Right. It's the real problem is in the
middle, right where there's reasonable dispute.
So so my first and primary recommendation is a void to the

(01:23:48):
extent possible creating these disputable person.
And perhaps even avoid a semi solution because once that's
done it's going to be so difficult to change our minds.
That's right, right. As soon as we create these
disputable persons, then we create the social semi solution
and then we possibly end up thinking that we know and we

(01:24:08):
don't actually know. And that could be, so to speak,
a local minimum from which it's hard to escape.
Though that's another reason to avoid the avoid this confusing
middle. OK, so almost run time to round
up, but let's go through your philosophical practices.
I mean, it emphasizes living thequestion and how does this apply
when the question is, is this AIsuffering?

(01:24:35):
That's a hard question to live the I think it's really
important not to think entirely abstractly about philosophical
questions, as you're saying. I think that broad philosophical
theses are really just skeletonsof views rather than really

(01:24:55):
fleshed out views. You kind of need to really sink
your teeth into examples, and the best way to sink your teeth
into real examples is to live that way, right?
So vague theories like maximize utility or act on the maxim you
can will to be universal law, those are almost empty, right?
It's like it's when you do example after example and try to

(01:25:17):
live that way that you actually put flesh on a philosophical
view, right? So I think that I haven't done
that yet. We haven't yet quite crossed
into this place where disputablepersons are being created or
being possibly created. And as soon as we do, we'll have
a much richer understanding of what the possibilities are and

(01:25:41):
how to avoid the excluded middleand all that.
So I, I feel like we're, we're still in, we're in the fog,
right? To get to what you were saying
about the very beginning of the book, right?
We're in the fog and I don't claim to be able to see more
than a few meters into the fog. And I anticipate that, you know,

(01:26:04):
in a decade or two will be a lotfurther along and people will
look back on my foggy book and they'll say, wow, that was
really abstract and lacked foresight.
And but I think that's the position we're all in.
We're going to go from the fog to the leapfrog.

(01:26:26):
That's right, we're leapfroggingthrough the fog.
Exactly. You, you've already answered
this. My, my, one of my last questions
was if we don't know in time, what does the responsible action
look like for science, philosophy and policy?
But you've, you've kind of touched on it.
Is there anything about that you'd like to and back further

(01:26:46):
before we close off? Yes.
So the design policy, the excluded middle is, is one of my
primary recommendations, but another one that I've been
working on and I've got a paper and draft on this with Jeff
Siebo, we call the emotional alignment design policy.
Though emotions I think tend to guide people much more than

(01:27:09):
abstract principles. And so I think we want to be
really careful when we design AIsystems that we get the users
emotional reactions right. Though to the extent we can see
a little bit into the fog, we want to, we want to create AI
systems that people have emotionally appropriate
reactions to, right? So for example, we don't want to

(01:27:30):
create AI companions that peopleattach to as much as they attach
to humans. Of course you can get
emotionally involved with an AI companion.
That's fine, maybe even healthy.We get emotionally involved in
fictions, We get emotionally involved in role-playing games.
There's there's nothing wrong with that, but there's always a
limit, right? You know, when the when the
movie is over, you put your tissues away.

(01:27:51):
You know, nobody really died, right?
When the role-playing game is over, you don't confuse that
with reality. There's got to be that same line
for AI systems, right? You got to we, we want to make
sure that people's reactions, ifwe're staying on the side of
these companions, are not persons, right?
We got to make sure that the reactions are appropriate to

(01:28:14):
fiction and role play and not the kind of reaction we have to
real persons. Because we don't want people say
spending money that they should be spending on dentistry for
their children for upgrade features for the AI companion
because they they can we emotionally react to it as
though it's a real person who's suffering, right?
We don't want that. So that's part of the emotional

(01:28:36):
aligned design policy, right? There's, it's got other aspects
too. I don't think we will have the
time to get really deeply into it.
But yeah. But I would combine that with
the design policy of the excluded medal is one of my 2
primary recommendations for ethical AI design.
You you close the book with a quiet hope.

(01:28:57):
You say even in the fog we can act with humility and care.
If future generations look back and say we get strong about AI
consciousness over or under attributing, what would make our
error forgivable? Yeah, well, humility and care,
that sounds good to me, right. I think when we look back on the

(01:29:23):
moral catastrophes of say, the Nazi era or US slavery or
apartheid or Dolan's purges in the Soviet Union or whatever,

(01:29:44):
it's pretty clear that there's alack of humility in care in all
of those, right? Which is part of what makes them
unforgivable. And if you do try to act with
humility and care, you're going to err on the side hopefully of,
I mean, you're, we're still going to make mistakes, but they
won't be the same kinds of unforgivable moral mistakes that

(01:30:10):
I've just discussed. Well, yeah, I think that's a
beautiful way to sort of conclude this, Eric.
It's such a, it's been such a pleasure, man.
Reading your work is always fun,it's always exciting when you
come up with new projects, new books, anything about AI,
unconsciousness you feel that wehaven't touched on, that you
would really like to make sure people do know and do get, do

(01:30:32):
listen to. I feel like we've covered it
pretty well. I mean, of course, there's many,
many more things we could talk about, but but I kind of like
how the arc of how this conversation has has gone.
So I think I'm ready to wrap. Up.
Thank you so much, Eric. It's been an absolute pleasure
to chat to you again. Really appreciate it.
Yes, thanks for having me on.

All Episodes

Will We Know When AI Becomes Conscious? The Epistemic Fog of Artificial Minds | Eric Schwitzgebel

Episode Transcript

Popular Podcasts

Stuff You Should Know

Medal of Honor: Stories of Courage

Dateline NBC

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}Will We Know When AI Becomes Conscious? The Epistemic Fog of Artificial Minds | Eric Schwitzgebel