May 2, 2023 • 37 mins
Liberty Vittert and Munther Dahleh dive into the world of augmented and virtual reality this month with Professor Fox Harrell. Harrell is Professor of Digital Media & AI in both the Comparative Media Studies Program and the Computer Science and Artificial Intelligence Laboratory at MIT. Is the world ready for what’s coming? Will augmented reality and virtual reality be a force for good or for evil, and what can you do to prepare yourself?
You can follow us on Twitter and Instagram @mitidss. Thanks for listening to Data Nation from the MIT Institute of Data Systems and Society.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
[MUSIC PLAYING]
(00:04):
Hello and welcometo Data Nation,
a podcast from MIT's Institutefor Data Systems and Society.
I'm Liberty Vittert,and I'm here
with my co-host, Munther Dahleh,the head of MIT's Institute
for Data Systems and Society.
In this episode, wetalk with Fox Harrell.
Fox is professor ofDigital Media and AI
in both the comparative mediastudies program and the Computer
(00:27):
Science and ArtificialIntelligence Laboratory at MIT.
Professor Harrell's researchinvestigates and innovates
ways that thecomputational world can mix
and layer with our actual world.
You hear the word "augmented"reality or augmentation
of reality.
We hear the word"virtual" reality.
Can you give me justa layman's version
(00:49):
of what are all thesedifferent realities?
What do they actually mean?
I'd be happy to share withyou a layperson's introduction
to this medium.
In some way, we all havevirtual extensions of ourselves,
because if you use asocial media profile,
you have a virtual self.
If you have any kindof e-commerce account,
you have a virtual self.
(01:10):
If you have any kindof avatar or character,
you have a virtual self.
But when it comes to augmentedreality, virtual reality,
extended reality, and so forth,I think about it as a spectrum.
The reason thatwe call the center
that I founded anddirected here at MIT,
the MIT Center forAdvanced Virtual Reality,
is because I'm interested inall of the kinds of technologies
(01:33):
that blend the physicalworld with the computer,
with thecomputationally-generated world.
And you could say each of thosetechnologies you mentioned,
does something similar.
So, for example, virtualreality blocks the world
and replaces itwith something else.
Augmented reality overlayssomething on top of the world.
(01:54):
Mixed reality is typicallythought of as augmented reality,
but handling occlusion.
So it looks like things areactually embedded in the world.
And extended realityis often used
as an umbrella term for allof those types of technology.
That's great.
Actually, that'snice positioning.
I feel like that's thefirst time I understand
what all these things mean.
(02:15):
Yeah, I agree with liberty.
And having heard theseexpressions, even from you, Fox,
I think this is the first timeyou actually define it for me,
and it's very helpful.
But then, given thatwe have this spectrum,
it's clear that there aredifferent applications
and different valuefor different aspects
of the spectrum.
A lot of time, we thinkof gaming as one way
(02:36):
where there is aspectof virtual reality,
maybe even augmented reality.
How big is the spectrum ofapplications of these ideas?
It's a great question.
So one way that I think about itis, first of all, just thinking
about computationalmedia more broadly,
because unlike some otherprior forms of media,
(02:58):
the computational medium, somepeople refer to it as a media
medium, that is, it can emulateprevious forms of media, that
is, we can watchcinema on the computer,
we can listen to radio onthe computer, and so forth.
We can play gamesthat were formerly
board games on the computer.
But it also has itsown characteristics.
And those characteristics,you could say, at the core,
(03:22):
is that, it's algorithmicand data structural,
and everything thatfalls out from that.
And so that's just, Iguess, an entry point
to say that the applicationsare very broad when you think
about it as a mediamedium, you could just
sit in a virtual realityspace and watch a screen
(03:44):
in front of you.
It wouldn't be the mostinnovative use of the medium.
In terms of gaming,like you mentioned,
I think some people thesedays are surprised just
to learn how far the gamingindustry has outpaced the film
industry globally, sportsindustry, globally,
in terms of revenue.
(04:05):
So it's just apervasive medium that
has massive uptake these days.
But increasingly, foreducation, for business,
for such a rangeof applications,
different types of virtualworlds are being used.
A
Lot of times, following theidea of computational media
and its data structureand algorithmic nature,
(04:27):
is the same contentcan be broadcast,
streamed, participatedin in different ways
on different platforms.
So the same content thatcould be experienced
through, say, a VRheadset could also
be experiencedthrough a mobile phone
and also experience throughyour browser, and so forth.
(04:50):
So when you thinkabout it that way,
it's just in massiveuse and pervasive.
Although, the uses don't alwaysmaximize every affordance,
just meaning that not everysingle use is going to be
an immersivevirtual world in 3D,
where you take into accountproprioception and the sense
of the user in space andall of these sort of things.
(05:14):
This is really interesting,Fox, because I'm
beginning to think aboutthe breadth by which
this mixed reality plays a veryimportant role in our society.
And I'm thinking ina very primitive way.
I was flipping throughmy TikTok the other day,
and there's all theseadvertisements about how
(05:36):
you actually look different.
You can run your filtersthrough your images
and you create an image ofyou, but a lot better than you.
And I think ofthat all the time.
So what is real?
What is the real, and what isaugmented, and what is virtual?
And AI is doingthis a lot to us.
(05:56):
You can frameyourself differently,
but it's stillyou in some level.
So what is real andwhat is virtual?
It's a great question.
And I'm trying hard toresist the low-hanging fruit
of the joke you mentioned,when you say it's like you,
but better than you.
I say that, yeah,many users find
(06:17):
that to be the case, although,I've not found the better
version of me yet.
[INAUDIBLE]
You're the lucky one.
I will interleave that kind ofjoke with a lack of humility
with the self-deprecatingone next,
just so you get a sense of thefull spectrum of personality.
But when you mention thisquestion of what is real,
(06:38):
of course, it's sucha profound question.
And one way thatI think about it
is like this, becausepeople sometimes
distinguish the virtualfrom the real and they say,
that was a virtual world,and this is the real world.
That was the virtual character,that's the real you, and so on.
(07:00):
And I don't think aboutit exactly that way,
because let's say that you'rein an online course for learning
computer science and you learnbreadth first search, depth
first search, and you begin tolearn more advanced concepts,
and so forth, then youstep out of that platform.
It's not as if you didn'tactually learned something.
(07:21):
You learned something, itjust was computationally
mediated the same way you couldsay that a conversation that you
have on the telephone isn'ta fake conversation just
because it's not in-person.
It's just mediatedthrough the telephone.
Similarly, if somebody hasan experience of harassment,
(07:42):
threat, and so forth online,mediated by the computer,
I wouldn't want to dismiss itto say something like, well,
that wasn't a real experience.
That was just avirtual experience.
Again, it was just mediatedthrough the computer.
So when I think aboutthe real and the virtual,
I tend to use the terms"virtual" and "physical world."
(08:05):
And I use physicalworld in place
of what people tend torefer to as the real world.
But there are so manymore nuanced ideas
about this kind of question.
So, for example, in JeanBaudrillard's book, Simulations,
he begins to talk about notionsthe like the hyper real.
(08:26):
And this is whereyou have experiences
in the physical world that areso based in the artificial,
whether that's because ofsynthesized, manufactured
environments, or advertising,or all of these things
that mediate modernlife and you begin
(08:46):
to lose that sense ofwhat feels like something
essential and authentic.
I mean, the examplesthat are given,
one often isCelebration Florida,
which is a city that is likeMain Street in Disneyland,
but it's this entire town.
(09:07):
It's a manufactured town.
And you can say, is thatreally a small town?
It didn't comeabout organically.
It was planned.
And so that sort of thing, hebegins to refer to as the real.
And it might be abit theoretical,
but I mention itjust because when
you talk about a termas loose reality,
you can also talk about theways that what we experience
(09:29):
in the physical world can feelmore or less real in a way.
And then you can alsodistinguish between what
happens in the virtual world.
To point this out, in my class,I have the students read both
about Celebration Florida, andthere is a Village Voice article
from the 1990s with a harrowingtitle by Julian Dibbell,
(09:53):
which is called ARape in Cyberspace.
And this is about a platformwhich is entirely text only,
everything is described.
And you'd describe what you'd,say, go north, go south, pick
up object, and so forth,but with other people there.
And somebody hadan artifact where
they could controlanother person's character
and they basically harassedthem in a horrible way.
(10:16):
And although it wasa virtual character,
they felt really disturbed bythe trauma of the experience.
And it became aninflection point
because people would say clearlysomething traumatic occurred,
but we can't conflatethis with trauma
in the physical worldat the same time
and say that they are the same.
(10:36):
So what does thismean, that there
was some psychologicaldamage that was inflicted?
And so I contrastthat experience
because it's somethingthat somebody certainly
experienced asreal with something
in the physical world thatfeels quite artificial
so the students can beginthinking about exactly that
question that youmentioned from day one.
I think that brings up suchan interesting concept of,
(11:01):
do we get programmed, in away, to perceive things that
aren't 100% real?
Whether it'ssomething artificial
in the physical world.
I put on a ton of makeup.
So I'm artificial inthe physical world.
Or I go on--
[INAUDIBLE], I'm shockedto hear you have a TikTok.
I think that's fabulous.
But to go on TikTokand I put a filter on.
(11:23):
I have no makeup on, butI put a full filter on,
and then I'm in thevirtual world, but someone
completely different.
Would you almost saythose are the same thing?
Everyone's fearof, we're getting
programmed toperceive things that
aren't real bythe virtual world,
is that not evennecessarily fair
because we're also programmed tosee things as not 100% real that
(11:44):
are in the physical world?
I would say that some of thesocial phenomena are the same,
and we could go back toseminal work in sociology,
such as Erving Goffman'swork on self-presentation
in everyday life, that is, howpeople manage the impressions
that others have of them.
And in the physicalworld, of course,
(12:04):
we are doing impressionmanagement frequently.
And people do the samething in virtual worlds.
When it's based on algorithmsand data structures,
then that allows you to dodifferent things than you
could in the physical world.
So for example,with virtual makeup,
the developer might have accessto when you purchased it,
(12:27):
from whom, or was it traded toyou from which online store.
There might be a lot ofmetadata around that.
You could tracktrends quite easily.
Somebody could changethe implementation of it
so that the samemetadata then refers
to a differently-appearingmakeup.
For example, you could make itdynamically change over time,
(12:48):
and say, processit algorithmically.
So there's a lot of differentthings on top of that technology
that you could do differentlythan in the physical world.
Although, of course, withphysical world makeup,
there would be some thingsthat you couldn't do in
the virtual worldvery easily that
have to do withtactile sensation,
maybe, with themethod of application,
(13:09):
with the blurring, blending.
It could be done, butit would take, maybe,
some more advanced techniques.
And so a lot ofmy work is looking
at the intersectionof issues, like what
persists from the physical worldin terms of social phenomena?
Because I do a lotof social simulation.
One of my friends, aprofessor at UC Santa Cruz,
(13:30):
likes to refer to itas social physics.
In the same way that games havephysics engines to model physics
relationships, and gravity,and those sorts of things,
this is a work that'smodeling social phenomena,
like social physics engines.
Then what also becomes newin those environments, what
new forms of experience, whetherempowering, disempowering,
(13:52):
and so forth, come aboutbecause of using the computer.
One key aspect of it is that,in the physical world, I mean,
we're interactingwith other people.
And all of the phenomena justfall out naturally from that.
Sometimes in thecomputational world,
people think thatjust because you
change the physical appearanceof an avatar, for instance,
(14:14):
then you will know whatthe experience of somebody
else in the world would be.
And I don't know why peoplemake that kind of mistake.
It's like if somebodyputs on, to continue
with your example,makeup and they
look like a different personin the physical world,
I think people are lesslikely to think that now they
know entirely what it's liketo be that other person now.
(14:36):
But in the computer, sometimespeople like to think,
just because you use anavatar of one gender,
or one ethnicity,or nationality,
you now somehow know whatit's like, like you've
walked in the shoes ofsomebody from that group.
And I feel like that's amisconception about the medium.
But this is actuallyreally interesting.
(14:58):
And we're segueing a little bitfrom images into, essentially,
social experiments.
I mean, you talk aboutsocial simulation, but what
about social experiments?
And in many ways,it's very expensive
and it's very hurtful to have anactual social experiment, where
physical people areinteracting with each other
because that's not that easy.
You cannot intervene.
(15:19):
You cannot do thingsthat cannot be done.
But you can do them inan augmented reality.
And both of us can beinteracting with the game
or interactingwith the platform.
And what you're highlighting,it's not always the case,
that we behave the same way.
And so I'm curiousto your thoughts
about using this mediumas a social experiment
and to what extent theinformation that we draw out
(15:42):
of it is biased or isnot necessarily valid.
I think about it in this way.
It's a kind of parable.
There is a story by theauthor Jorge Luis Borges
called On Exactitude in Science.
If people know this author,he writes these parables just
(16:04):
to think about thenuances of life.
There always sometricky outlook on life.
So this is oneabout a society that
creates a map thathas a one to one
relationship withthe physical world.
And so you couldthink, if there was
a map that was like this, then,essentially, it's large enough,
it would cover the world.
(16:25):
And so it's animpossibility, in a way,
to create a map thathas complete fidelity
to the actual physical world.
Because then, in a way,almost, it would be the world.
That's the idea of the parable.
I mention it justbecause I think
when we have these simulationswithin virtual worlds
(16:46):
that are like this, it'ssimilar to creating a map.
If you create a map of theglobe, that's a flat map.
There's going to be somedistortion that takes place
or there's going to be someelements that are omitted.
Do you show certaingeographical features?
Do you show national features?
And so forth.
And I think of the process ofdesigning these experiments
(17:10):
as a similar kind of mapping.
You have to decidewhat you're going
to map from the physicalworld phenomenon
to the simulated phenomenon.
And there are disciplinedways that we can do this.
I use an approach, which iscalled isomorphic semiotics
sometimes.
It can be used informallyor very formally.
But essentially, it'slooking at the sorts
of things that are being modeledthat is like the data types,
(17:34):
looking at the ordering, therelationship of them to one
another, and so forth,to look at a structure
of the social phenomenon andthen think through, carefully,
what we want to preservein the simulation.
And that decision ofwhat we want to preserve
is typically driven, in the caseof social science simulation,
by looking for the latestsocial science theories
(17:57):
in that particular area.
So when we did work on modelingsexism in the workplace,
we looked at a model, which iscalled the ambivalent sexism
inventory.
And we thought, what dowe want to incorporate
and what do wewant to leave out?
Because it's such a pervasivephenomenon and, in some ways,
so well-known, but notwell-explained, and not
(18:18):
well-recognized, sometimes,just in terms of the outcomes
not being commensurate withthe injuries that are caused.
So we chose thisparticular model
because it highlighteda few phenomena.
Hostile sexism, whichis what is covered,
# say, for example, in the#MeToo movement, but also
(18:39):
forms of everyday sexism, likecertain paternalistic attitudes,
complementary genderdifferentiation,
like assuming peoplewill be good at one
skill or another,not good at one
skill or another becauseof gender in part.
And so we wanted to take thatmodel and then think, how can we
preserve that in the simulationso that people can recognize,
name, see the systematicityin those kinds of phenomena?
(19:02):
So that's a typicalway that we work.
And I think if we work it withdiscipline and care in this way,
then you're creating areasonable distillation
of the phenomenon to study.
If you just do somethingas simplistic as, say, give
somebody an avatar that hasa differently-appearing body,
(19:25):
then you're not modeling any ofthe experiences, the situations,
the systematicity ofthat kind of experience,
then I think it will be a verypoor simulation, most likely.
If you're looking tostudy an experiment
to study a particularphenomenon,
I think you actually haveto model that phenomenon,
and not just expect it tofall out of a simulation.
(19:47):
So that's a pretty nuancedanswer to your question.
I think this medium can bequite important for experiments,
but we have to be very carefulin how we conduct the simulation
and run the experiments.
Let me just follow upquickly on this question.
So this is Grayscale thatyou're talking about?
Yes.
Yeah, but I wanted to understandthat a little bit better,
(20:07):
since I haven'tused the platform.
So was the definition of theambivalent sexism something
that emerged from thenarrative or that was built in
and narratives were used tosimulate that sort of outcomes
or neither?
So Chimeria:Grayscale, it's aninteractive narrative that was
(20:29):
built using a patented platformthat came out of my lab
and my research.
And that platformis used for, one,
generating interactivenarratives,
and then also modelinghow people are
categorized in a nuanced way.
A lot of computational systemscan model categorization,
(20:51):
but usually, it's justa binary, like a set
that you're either in orout, some set of fields that
determine whetheryou're in the category
or outside of the category.
What we know from the cognitivescience work on categorization
is that, in practice, it doesn'twork that way, that we have
degrees of membershipin categories,
multiple membershipin categories
(21:13):
that our categorymemberships fluctuate
and change over time, overthe course of our lives,
and so forth.
And so Chimera can model someof those types of phenomena.
In this case, theprotagonist works
as an HR manager in acompany called Grayscale,
and they're faced withdifferent dilemmas at work.
(21:34):
And those dilemmas canrange from somebody
comparing two differentresumes of job applicants
and usingthinly-veiled language.
They look like a realgo-getter and a team player,
dynamic person, orbeautiful CV, they
look like they'll bea real people person.
(21:54):
And one might be theresume of someone
with a name that'sassumed to be male
and the other one isassumed to be female.
We have a range of differentkind of circumstances, people
trying to regulatewhat other people are
wearing in the workplaceand these sorts of things.
I see.
And as you make thosechoices, each of those choices
is aligned with the categoriesin the ambivalent sexism
(22:17):
inventory.
And they move you, as theplayer, as this manager,
in and out of thosemultiple categories.
And then basedupon the categories
that you're exhibiting traitsof, if you're seen as supporting
ambivalent sexismin some forms, you
have different outcomesand responses to the story.
And at the very end, youget a performance review.
(22:40):
And that performancereview is narrative,
and it's not just meantto be a boring training
tool that sometimespeople encounter,
but an interestingstory, where you can then
reflect on what happened.
You might even have somethinglike, well, things seem
to be getting better there.
People are pretty happy.
But if this doesn'tpersist, we might have
(23:01):
to go back to the old ways.
So it might make you reflect onproductivity within the company
versus supportinganti-sexist measures
and how they're judged againstone another, and so forth.
That kind ofnarrative, by the way,
it plays out likean email system.
It looks almost just likean email system, just a bit
aestheticized and streamlined.
(23:22):
And you're selecting howyou respond to emails.
That's called anepistolary narrative.
Epistolary narrativesare those that
are told through differentfragments of communication,
like diary entries, journalentries, emails, letters,
those sort of things.
So essentially, it's aninteractive epistolary narrative
that you're immersed in.
And the idea is that shouldbe more engaging, narratively,
(23:43):
than something which is justa training tool that tells you
how to think becauseyou're actually
exploring the phenomenon.
And then withdebriefing, afterwards,
after you're donewith the experience,
it can highlight andname the phenomena
that you have just experienced.
I have to say, itsounds amazing.
I just the sexual harassmenttraining at work thing.
(24:04):
You watch two videos.
It's the most boring thing ever.
You click through the answers.
You answer some multiplechoice questions
that anyone with any commonsense would get 100% on.
And you get a score at the endof this miserable click through.
And I learned nothing,and it was pointless.
And so this sounds like1,000 times better.
But is there any wayto actually measure it?
(24:28):
Is there any way tomeasure that this system,
this Grayscaleexperiment, is better
than the traditionalharassment training
videos with multiplechoice questions
that you click through at work?
Well, you do runstudies on this.
And so, in that case, we werelooking for a phenomenon, called
perspective transformation,that comes from the learning
sciences.
(24:48):
And we have various waysof reading user data.
It can range frombiometrics based
on galvanic skinresponse, and heart rate,
and those sort of things.
We have surveys that wecan use that are validated,
instruments created bysocial scientists, and more.
A lot of our work, we'verelied on these surveys.
(25:08):
And in the case of someof this work, perspective
transformation,we're looking at,
did people challenge theirprior presuppositions
that they came in with?
And the learningsciences theory on this,
it suggests that thereare multiple stages
of prospective transformation,and essentially, we're
looking to see if peopletransform their perspectives
(25:29):
from when they began.
And I mean, there area lot of instruments.
We've done work ondiscrimination in schools.
So my former PhD studentwho graduated a year ago,
Danielle Olsen, thesis was on VRsimulation of racist experiences
in education.
And that used a model calledracial and ethnic socialization
(25:54):
model.
And in that case,it both assesses
users to see whatcategory of socialization
that they've beenraised through, that is,
through parents, through media,through peers, and so forth.
There were fourcategories in that case.
One is called a colorblindcategory, where people say,
(26:14):
I don't see race.
It doesn't exist.
We shouldn't talk about it.
There's another category,which is distrust
of other people,which fortunately is
the one that you see the least.
But it is out there.
I mean, distrust of peoplefrom different groups.
You have prepared for bias.
Sometimes people thinkabout it these days
as related to the talkas an example of it.
(26:35):
That's the talk that people inunderrepresented or oppressed
groups in society might saygive their children when
having to potentially deal with,say, biased authority figures,
and say, you mightexperience this bias.
Here's how you might want tocomport yourself, and so forth.
And then the final one ispride and legacy reinforcement,
(26:56):
which is a social acceptance andpride of not only your group,
but the groups ofothers, and looking
at the positive characteristicsthat might exist
within those different groups.
And so we essentiallywere able to,
through this interactiveexperience, where
the user interactsthrough selecting gestures
for the character, assess howthey were socialized to think
(27:18):
about race from the experience.
And then it can alsobe an intervention
to help them then reflect on andhelp with a kind of perspective
change based upon themthinking through the issues
as they go throughthe simulation.
Yeah, so I have one lastquestion from my side,
and it has to do a littlebit with the metaverse.
(27:41):
As you know, atMIT, a lot of people
have been talking aboutmetaverse and the creation of, I
would call it, augmented realityto navigate the complex world
of the internet, and so forth.
Do you think that this isgoing to help us understand
the world a littlebit better, or is it
going to isolate us evenmore from human beings
(28:04):
and the rest of the world?
And I mentionedthis as an example,
where recently, we visited Peruand we went to Machu Picchu.
And it's a trek,and it's actually
complicated to getto Machu Picchu.
And the question is, I couldhave done a metaverse on this
and got some ideaof the whole place.
Is that a good alternative?
How is that affecting us ashuman beings focusing so much
(28:27):
more about thisaugmented reality?
Actually, I willclarify, just to say,
because I don't think it'sjust augmented reality.
I think it's somethinglike virtual worlds
with social media, affordances,and networked computing
all coming together.
And you could think aboutthe metaverse as something
like this.
And I think that, like a lot ofthings, it depends on the use.
(28:55):
My PhD advisor, JosephGauguin, was staunchly
against what's known astechnological determinism,
the idea that any technologyis a unilateral force that
will shape thefuture of society,
for better or for worse.
Any technological deterministstatement is false,
(29:17):
is what he used to say.
And people would argue,they'd say, but certainly,
the cell phone,that's changed life.
Or people might say, nowadays,certainly, the metaverse,
that has changed society.
And in that case, therewould be missing the key word
in this, which is theunilateral side of it.
And so the alternativeway that I think about it,
(29:39):
I think it's in line withthinkers like sociologists Bruno
Latour and others,is it's a ecology.
There's a network ofdifferent forces at work
that includes people's, say,everyday physical relationships,
the relationships online,the technology itself,
the documentationof the technology,
(30:00):
the way it's reported onin the media, and so forth.
You could say, the effectscome from that whole network
of forces.
And I think that's what makesit a complex question to answer.
But if I wanted just to givea personal response, just
my own feeling about it,is that, one, I think,
(30:20):
if the experiences people haveultimately pushed people back
to the physical world andphysical world relationships,
I think there's somethingabout not making
it a surrogate for physicalworld relationships
and experiences,but something that
loops you back towards thosekind of experiences and needs.
(30:42):
If it just absorbs you intosomething which is simulated,
there's just so many dangers.
It can give the illusion ofthe experiences you would have
in the physical worldwith all of that nuance,
but really, it'shighly regulated,
created by a system ofindustry, and so forth.
So the nutshellversion is I would
say that there's quitea danger that there,
(31:04):
although, if designed incareful and sensitive ways,
there also are profoundlypositive opportunities that
are available.
And hopefully, manyof the designers
are cognitive of thisissue when they design
aspects of the metaverse.
I hope, increasingly so.
I mean, it just dependson the designers.
(31:26):
But they're gettingto be some practices.
I guess, you could say it'slike in other areas of software
engineering thatthere are many cases
where people only thinkabout individual users
or only think about, let's say,the uptake of the platform,
and so forth.
But there are strands ofhuman-computer interaction
and softwareengineering, where people
(31:47):
think about social impact, buildthat into the process of design,
work with variousstakeholder groups,
involved the stakeholdergroups in the design process
and evaluationprocess, and so forth.
And I think it's similar.
We begin to see some worksthat are developed hand-in-hand
with community members.
Just now, I'm on a grantwith researchers in Canada,
(32:09):
and also, I mean,just internationally
on indigenous cultures andartificial intelligence,
and just thinkingabout the impacts
on different global indigenousgroups, and so forth.
And so I think that there are anincreasing number of initiatives
that take this kindof work into account.
I think that's maybe one ofthe hallmarks of the Institute
for Data Systemsand Society at MIT,
(32:30):
is just the factthat people are both
looking at those socialfactors, as well as
the information and dataeconomy side of the work.
If you use that as aparallel, you could ask,
how many departments aredoing something similar that
take into account thetechnical and the social,
(32:50):
and how many are focusedprimarily on just the end
use in a more technical sense?
And that might be parallelto other types of software,
like entertainmentand metaverse, too.
Thank you.
So Fox, this is totally outthere and your personal opinion
on this, but in terms ofwhat is the future of this,
(33:11):
is there a world in whicheveryone walking around
has virtual realitygoggles on and everyone
is living in this simulatedworld that's not physical, or is
this just the stuff of movies?
I think if we extrapolate fromwhere we are now, as everybody
(33:34):
who has even a few yearsago, we call it a smartphone,
but now it feels almostredundant to say smartphone
because everybodyjust has a phone that
is of this sort carrying arounda computer in your pocket.
And so I think the idea thatpeople will be networked,
that people have virtualsurrogates for themselves,
(33:57):
different worlds toexplore, and ways
to engage withcomputational media, that
certainly seems to be the case.
Whether that lookslike something
like a headset orglasses, and so on,
I think that wouldremain to be seen.
(34:17):
I think it will happen ina seamless kind of way,
like the integration ofthe phone and the camera.
At first, people thoughtthese are just two devices.
Why would they haveto be paired together?
And incrementally, peoplebegan to be used more and more
(34:39):
to having different formsof mediation of life
that you're carryingaround with you.
But I think there's a problemwith the headset, in the idea
that, again, blockingthe physical world
and then replacing itwith something else,
but overlay in ways thatseem banal initially.
(35:00):
But the same way that we scana grocery store card and that
connects us to a nationalor global network of data,
and then that datatransforms our experience
of the store because it guideswhich products are going
to be placed therenext, and so forth,
it's very easy to imaginepicking up your phone
(35:22):
and now you see informationabout those vegetables
or whatever is in front of you.
And then just incrementally,a bit more and more.
Maybe advertisingcreeps in, and so forth.
So I think it willhappen more like that,
not a spectacle, in which,within a few years everybody
is immersed in thisglobal illusion.
(35:45):
But we'll become less andless sensitized to the way
that data is overlaid onour daily experiences.
And it might look not somuch like being immersed
in some exciting virtual world,like the Holodeck in Star
Trek, The NextGeneration, but it
(36:06):
will be more likewhat we experience
of going online now withour phones in the store
or walking around, where you'rejust a little less connected
to where you are.
You have access to a lot ofdifferent types of information.
You want to know a little bitabout that musical instrument
you're going to buy.
So let me look it up.
But now, instead, youjust see it laid on top.
(36:28):
I could imagine things justgoing in that direction
if I were to extrapolate ina fanciful way just based
on where we are now.
Thank you so muchfor doing this.
We know how busy you are, so wereally can't thank you enough.
Thanks, Fox.
Every time I listen toyou, I learn something more
about that word.
Thanks.
(36:48):
It was absolutely my pleasure.
Thank you for listening to thismonth's episode of Data Nation.
You can get more informationand listen to previous episodes
at our website,idss.mit.edu or follow us
on Twitter andInstagram @MITIDSS,
@LibertyVittert,and @MuntherDahleh.
If you like thispodcast, please don't
(37:08):
forget to leave us areview on Spotify, Apple,
or wherever youget your podcasts.
Thanks for listeningto Data Nation
from the MIT Institute ofData Systems and Society.
[MUSIC PLAYING]
(00:04):
Hello and welcometo Data Nation,
a podcast from MIT's Institutefor Data Systems and Society.
I'm Liberty Vittert,and I'm here
with my co-host, Munther Dahleh,the head of MIT's Institute
for Data Systems and Society.
In this episode, wetalk with Fox Harrell.
Fox is professor ofDigital Media and AI
in both the comparative mediastudies program and the Computer
(00:27):
Science and ArtificialIntelligence Laboratory at MIT.
Professor Harrell's researchinvestigates and innovates
ways that thecomputational world can mix
and layer with our actual world.
You hear the word "augmented"reality or augmentation
of reality.
We hear the word"virtual" reality.
Can you give me justa layman's version
(00:49):
of what are all thesedifferent realities?
What do they actually mean?
I'd be happy to share withyou a layperson's introduction
to this medium.
In some way, we all havevirtual extensions of ourselves,
because if you use asocial media profile,
you have a virtual self.
If you have any kindof e-commerce account,
you have a virtual self.
(01:10):
If you have any kindof avatar or character,
you have a virtual self.
But when it comes to augmentedreality, virtual reality,
extended reality, and so forth,I think about it as a spectrum.
The reason thatwe call the center
that I founded anddirected here at MIT,
the MIT Center forAdvanced Virtual Reality,
is because I'm interested inall of the kinds of technologies
(01:33):
that blend the physicalworld with the computer,
with thecomputationally-generated world.
And you could say each of thosetechnologies you mentioned,
does something similar.
So, for example, virtualreality blocks the world
and replaces itwith something else.
Augmented reality overlayssomething on top of the world.
(01:54):
Mixed reality is typicallythought of as augmented reality,
but handling occlusion.
So it looks like things areactually embedded in the world.
And extended realityis often used
as an umbrella term for allof those types of technology.
That's great.
Actually, that'snice positioning.
I feel like that's thefirst time I understand
what all these things mean.
(02:15):
Yeah, I agree with liberty.
And having heard theseexpressions, even from you, Fox,
I think this is the first timeyou actually define it for me,
and it's very helpful.
But then, given thatwe have this spectrum,
it's clear that there aredifferent applications
and different valuefor different aspects
of the spectrum.
A lot of time, we thinkof gaming as one way
(02:36):
where there is aspectof virtual reality,
maybe even augmented reality.
How big is the spectrum ofapplications of these ideas?
It's a great question.
So one way that I think about itis, first of all, just thinking
about computationalmedia more broadly,
because unlike some otherprior forms of media,
(02:58):
the computational medium, somepeople refer to it as a media
medium, that is, it can emulateprevious forms of media, that
is, we can watchcinema on the computer,
we can listen to radio onthe computer, and so forth.
We can play gamesthat were formerly
board games on the computer.
But it also has itsown characteristics.
And those characteristics,you could say, at the core,
(03:22):
is that, it's algorithmicand data structural,
and everything thatfalls out from that.
And so that's just, Iguess, an entry point
to say that the applicationsare very broad when you think
about it as a mediamedium, you could just
sit in a virtual realityspace and watch a screen
(03:44):
in front of you.
It wouldn't be the mostinnovative use of the medium.
In terms of gaming,like you mentioned,
I think some people thesedays are surprised just
to learn how far the gamingindustry has outpaced the film
industry globally, sportsindustry, globally,
in terms of revenue.
(04:05):
So it's just apervasive medium that
has massive uptake these days.
But increasingly, foreducation, for business,
for such a rangeof applications,
different types of virtualworlds are being used.
A
Lot of times, following theidea of computational media
and its data structureand algorithmic nature,
(04:27):
is the same contentcan be broadcast,
streamed, participatedin in different ways
on different platforms.
So the same content thatcould be experienced
through, say, a VRheadset could also
be experiencedthrough a mobile phone
and also experience throughyour browser, and so forth.
(04:50):
So when you thinkabout it that way,
it's just in massiveuse and pervasive.
Although, the uses don't alwaysmaximize every affordance,
just meaning that not everysingle use is going to be
an immersivevirtual world in 3D,
where you take into accountproprioception and the sense
of the user in space andall of these sort of things.
(05:14):
This is really interesting,Fox, because I'm
beginning to think aboutthe breadth by which
this mixed reality plays a veryimportant role in our society.
And I'm thinking ina very primitive way.
I was flipping throughmy TikTok the other day,
and there's all theseadvertisements about how
(05:36):
you actually look different.
You can run your filtersthrough your images
and you create an image ofyou, but a lot better than you.
And I think ofthat all the time.
So what is real?
What is the real, and what isaugmented, and what is virtual?
And AI is doingthis a lot to us.
(05:56):
You can frameyourself differently,
but it's stillyou in some level.
So what is real andwhat is virtual?
It's a great question.
And I'm trying hard toresist the low-hanging fruit
of the joke you mentioned,when you say it's like you,
but better than you.
I say that, yeah,many users find
(06:17):
that to be the case, although,I've not found the better
version of me yet.
[INAUDIBLE]
You're the lucky one.
I will interleave that kind ofjoke with a lack of humility
with the self-deprecatingone next,
just so you get a sense of thefull spectrum of personality.
But when you mention thisquestion of what is real,
(06:38):
of course, it's sucha profound question.
And one way thatI think about it
is like this, becausepeople sometimes
distinguish the virtualfrom the real and they say,
that was a virtual world,and this is the real world.
That was the virtual character,that's the real you, and so on.
(07:00):
And I don't think aboutit exactly that way,
because let's say that you'rein an online course for learning
computer science and you learnbreadth first search, depth
first search, and you begin tolearn more advanced concepts,
and so forth, then youstep out of that platform.
It's not as if you didn'tactually learned something.
(07:21):
You learned something, itjust was computationally
mediated the same way you couldsay that a conversation that you
have on the telephone isn'ta fake conversation just
because it's not in-person.
It's just mediatedthrough the telephone.
Similarly, if somebody hasan experience of harassment,
(07:42):
threat, and so forth online,mediated by the computer,
I wouldn't want to dismiss itto say something like, well,
that wasn't a real experience.
That was just avirtual experience.
Again, it was just mediatedthrough the computer.
So when I think aboutthe real and the virtual,
I tend to use the terms"virtual" and "physical world."
(08:05):
And I use physicalworld in place
of what people tend torefer to as the real world.
But there are so manymore nuanced ideas
about this kind of question.
So, for example, in JeanBaudrillard's book, Simulations,
he begins to talk about notionsthe like the hyper real.
(08:26):
And this is whereyou have experiences
in the physical world that areso based in the artificial,
whether that's because ofsynthesized, manufactured
environments, or advertising,or all of these things
that mediate modernlife and you begin
(08:46):
to lose that sense ofwhat feels like something
essential and authentic.
I mean, the examplesthat are given,
one often isCelebration Florida,
which is a city that is likeMain Street in Disneyland,
but it's this entire town.
(09:07):
It's a manufactured town.
And you can say, is thatreally a small town?
It didn't comeabout organically.
It was planned.
And so that sort of thing, hebegins to refer to as the real.
And it might be abit theoretical,
but I mention itjust because when
you talk about a termas loose reality,
you can also talk about theways that what we experience
(09:29):
in the physical world can feelmore or less real in a way.
And then you can alsodistinguish between what
happens in the virtual world.
To point this out, in my class,I have the students read both
about Celebration Florida, andthere is a Village Voice article
from the 1990s with a harrowingtitle by Julian Dibbell,
(09:53):
which is called ARape in Cyberspace.
And this is about a platformwhich is entirely text only,
everything is described.
And you'd describe what you'd,say, go north, go south, pick
up object, and so forth,but with other people there.
And somebody hadan artifact where
they could controlanother person's character
and they basically harassedthem in a horrible way.
(10:16):
And although it wasa virtual character,
they felt really disturbed bythe trauma of the experience.
And it became aninflection point
because people would say clearlysomething traumatic occurred,
but we can't conflatethis with trauma
in the physical worldat the same time
and say that they are the same.
(10:36):
So what does thismean, that there
was some psychologicaldamage that was inflicted?
And so I contrastthat experience
because it's somethingthat somebody certainly
experienced asreal with something
in the physical world thatfeels quite artificial
so the students can beginthinking about exactly that
question that youmentioned from day one.
I think that brings up suchan interesting concept of,
(11:01):
do we get programmed, in away, to perceive things that
aren't 100% real?
Whether it'ssomething artificial
in the physical world.
I put on a ton of makeup.
So I'm artificial inthe physical world.
Or I go on--
[INAUDIBLE], I'm shockedto hear you have a TikTok.
I think that's fabulous.
But to go on TikTokand I put a filter on.
(11:23):
I have no makeup on, butI put a full filter on,
and then I'm in thevirtual world, but someone
completely different.
Would you almost saythose are the same thing?
Everyone's fearof, we're getting
programmed toperceive things that
aren't real bythe virtual world,
is that not evennecessarily fair
because we're also programmed tosee things as not 100% real that
(11:44):
are in the physical world?
I would say that some of thesocial phenomena are the same,
and we could go back toseminal work in sociology,
such as Erving Goffman'swork on self-presentation
in everyday life, that is, howpeople manage the impressions
that others have of them.
And in the physicalworld, of course,
(12:04):
we are doing impressionmanagement frequently.
And people do the samething in virtual worlds.
When it's based on algorithmsand data structures,
then that allows you to dodifferent things than you
could in the physical world.
So for example,with virtual makeup,
the developer might have accessto when you purchased it,
(12:27):
from whom, or was it traded toyou from which online store.
There might be a lot ofmetadata around that.
You could tracktrends quite easily.
Somebody could changethe implementation of it
so that the samemetadata then refers
to a differently-appearingmakeup.
For example, you could make itdynamically change over time,
(12:48):
and say, processit algorithmically.
So there's a lot of differentthings on top of that technology
that you could do differentlythan in the physical world.
Although, of course, withphysical world makeup,
there would be some thingsthat you couldn't do in
the virtual worldvery easily that
have to do withtactile sensation,
maybe, with themethod of application,
(13:09):
with the blurring, blending.
It could be done, butit would take, maybe,
some more advanced techniques.
And so a lot ofmy work is looking
at the intersectionof issues, like what
persists from the physical worldin terms of social phenomena?
Because I do a lotof social simulation.
One of my friends, aprofessor at UC Santa Cruz,
(13:30):
likes to refer to itas social physics.
In the same way that games havephysics engines to model physics
relationships, and gravity,and those sorts of things,
this is a work that'smodeling social phenomena,
like social physics engines.
Then what also becomes newin those environments, what
new forms of experience, whetherempowering, disempowering,
(13:52):
and so forth, come aboutbecause of using the computer.
One key aspect of it is that,in the physical world, I mean,
we're interactingwith other people.
And all of the phenomena justfall out naturally from that.
Sometimes in thecomputational world,
people think thatjust because you
change the physical appearanceof an avatar, for instance,
(14:14):
then you will know whatthe experience of somebody
else in the world would be.
And I don't know why peoplemake that kind of mistake.
It's like if somebodyputs on, to continue
with your example,makeup and they
look like a different personin the physical world,
I think people are lesslikely to think that now they
know entirely what it's liketo be that other person now.
(14:36):
But in the computer, sometimespeople like to think,
just because you use anavatar of one gender,
or one ethnicity,or nationality,
you now somehow know whatit's like, like you've
walked in the shoes ofsomebody from that group.
And I feel like that's amisconception about the medium.
But this is actuallyreally interesting.
(14:58):
And we're segueing a little bitfrom images into, essentially,
social experiments.
I mean, you talk aboutsocial simulation, but what
about social experiments?
And in many ways,it's very expensive
and it's very hurtful to have anactual social experiment, where
physical people areinteracting with each other
because that's not that easy.
You cannot intervene.
(15:19):
You cannot do thingsthat cannot be done.
But you can do them inan augmented reality.
And both of us can beinteracting with the game
or interactingwith the platform.
And what you're highlighting,it's not always the case,
that we behave the same way.
And so I'm curiousto your thoughts
about using this mediumas a social experiment
and to what extent theinformation that we draw out
(15:42):
of it is biased or isnot necessarily valid.
I think about it in this way.
It's a kind of parable.
There is a story by theauthor Jorge Luis Borges
called On Exactitude in Science.
If people know this author,he writes these parables just
(16:04):
to think about thenuances of life.
There always sometricky outlook on life.
So this is oneabout a society that
creates a map thathas a one to one
relationship withthe physical world.
And so you couldthink, if there was
a map that was like this, then,essentially, it's large enough,
it would cover the world.
(16:25):
And so it's animpossibility, in a way,
to create a map thathas complete fidelity
to the actual physical world.
Because then, in a way,almost, it would be the world.
That's the idea of the parable.
I mention it justbecause I think
when we have these simulationswithin virtual worlds
(16:46):
that are like this, it'ssimilar to creating a map.
If you create a map of theglobe, that's a flat map.
There's going to be somedistortion that takes place
or there's going to be someelements that are omitted.
Do you show certaingeographical features?
Do you show national features?
And so forth.
And I think of the process ofdesigning these experiments
(17:10):
as a similar kind of mapping.
You have to decidewhat you're going
to map from the physicalworld phenomenon
to the simulated phenomenon.
And there are disciplinedways that we can do this.
I use an approach, which iscalled isomorphic semiotics
sometimes.
It can be used informallyor very formally.
But essentially, it'slooking at the sorts
of things that are being modeledthat is like the data types,
(17:34):
looking at the ordering, therelationship of them to one
another, and so forth,to look at a structure
of the social phenomenon andthen think through, carefully,
what we want to preservein the simulation.
And that decision ofwhat we want to preserve
is typically driven, in the caseof social science simulation,
by looking for the latestsocial science theories
(17:57):
in that particular area.
So when we did work on modelingsexism in the workplace,
we looked at a model, which iscalled the ambivalent sexism
inventory.
And we thought, what dowe want to incorporate
and what do wewant to leave out?
Because it's such a pervasivephenomenon and, in some ways,
so well-known, but notwell-explained, and not
(18:18):
well-recognized, sometimes,just in terms of the outcomes
not being commensurate withthe injuries that are caused.
So we chose thisparticular model
because it highlighteda few phenomena.
Hostile sexism, whichis what is covered,
# say, for example, in the#MeToo movement, but also
(18:39):
forms of everyday sexism, likecertain paternalistic attitudes,
complementary genderdifferentiation,
like assuming peoplewill be good at one
skill or another,not good at one
skill or another becauseof gender in part.
And so we wanted to take thatmodel and then think, how can we
preserve that in the simulationso that people can recognize,
name, see the systematicityin those kinds of phenomena?
(19:02):
So that's a typicalway that we work.
And I think if we work it withdiscipline and care in this way,
then you're creating areasonable distillation
of the phenomenon to study.
If you just do somethingas simplistic as, say, give
somebody an avatar that hasa differently-appearing body,
(19:25):
then you're not modeling any ofthe experiences, the situations,
the systematicity ofthat kind of experience,
then I think it will be a verypoor simulation, most likely.
If you're looking tostudy an experiment
to study a particularphenomenon,
I think you actually haveto model that phenomenon,
and not just expect it tofall out of a simulation.
(19:47):
So that's a pretty nuancedanswer to your question.
I think this medium can bequite important for experiments,
but we have to be very carefulin how we conduct the simulation
and run the experiments.
Let me just follow upquickly on this question.
So this is Grayscale thatyou're talking about?
Yes.
Yeah, but I wanted to understandthat a little bit better,
(20:07):
since I haven'tused the platform.
So was the definition of theambivalent sexism something
that emerged from thenarrative or that was built in
and narratives were used tosimulate that sort of outcomes
or neither?
So Chimeria:Grayscale, it's aninteractive narrative that was
(20:29):
built using a patented platformthat came out of my lab
and my research.
And that platformis used for, one,
generating interactivenarratives,
and then also modelinghow people are
categorized in a nuanced way.
A lot of computational systemscan model categorization,
(20:51):
but usually, it's justa binary, like a set
that you're either in orout, some set of fields that
determine whetheryou're in the category
or outside of the category.
What we know from the cognitivescience work on categorization
is that, in practice, it doesn'twork that way, that we have
degrees of membershipin categories,
multiple membershipin categories
(21:13):
that our categorymemberships fluctuate
and change over time, overthe course of our lives,
and so forth.
And so Chimera can model someof those types of phenomena.
In this case, theprotagonist works
as an HR manager in acompany called Grayscale,
and they're faced withdifferent dilemmas at work.
(21:34):
And those dilemmas canrange from somebody
comparing two differentresumes of job applicants
and usingthinly-veiled language.
They look like a realgo-getter and a team player,
dynamic person, orbeautiful CV, they
look like they'll bea real people person.
(21:54):
And one might be theresume of someone
with a name that'sassumed to be male
and the other one isassumed to be female.
We have a range of differentkind of circumstances, people
trying to regulatewhat other people are
wearing in the workplaceand these sorts of things.
I see.
And as you make thosechoices, each of those choices
is aligned with the categoriesin the ambivalent sexism
(22:17):
inventory.
And they move you, as theplayer, as this manager,
in and out of thosemultiple categories.
And then basedupon the categories
that you're exhibiting traitsof, if you're seen as supporting
ambivalent sexismin some forms, you
have different outcomesand responses to the story.
And at the very end, youget a performance review.
(22:40):
And that performancereview is narrative,
and it's not just meantto be a boring training
tool that sometimespeople encounter,
but an interestingstory, where you can then
reflect on what happened.
You might even have somethinglike, well, things seem
to be getting better there.
People are pretty happy.
But if this doesn'tpersist, we might have
(23:01):
to go back to the old ways.
So it might make you reflect onproductivity within the company
versus supportinganti-sexist measures
and how they're judged againstone another, and so forth.
That kind ofnarrative, by the way,
it plays out likean email system.
It looks almost just likean email system, just a bit
aestheticized and streamlined.
(23:22):
And you're selecting howyou respond to emails.
That's called anepistolary narrative.
Epistolary narrativesare those that
are told through differentfragments of communication,
like diary entries, journalentries, emails, letters,
those sort of things.
So essentially, it's aninteractive epistolary narrative
that you're immersed in.
And the idea is that shouldbe more engaging, narratively,
(23:43):
than something which is justa training tool that tells you
how to think becauseyou're actually
exploring the phenomenon.
And then withdebriefing, afterwards,
after you're donewith the experience,
it can highlight andname the phenomena
that you have just experienced.
I have to say, itsounds amazing.
I just the sexual harassmenttraining at work thing.
(24:04):
You watch two videos.
It's the most boring thing ever.
You click through the answers.
You answer some multiplechoice questions
that anyone with any commonsense would get 100% on.
And you get a score at the endof this miserable click through.
And I learned nothing,and it was pointless.
And so this sounds like1,000 times better.
But is there any wayto actually measure it?
(24:28):
Is there any way tomeasure that this system,
this Grayscaleexperiment, is better
than the traditionalharassment training
videos with multiplechoice questions
that you click through at work?
Well, you do runstudies on this.
And so, in that case, we werelooking for a phenomenon, called
perspective transformation,that comes from the learning
sciences.
(24:48):
And we have various waysof reading user data.
It can range frombiometrics based
on galvanic skinresponse, and heart rate,
and those sort of things.
We have surveys that wecan use that are validated,
instruments created bysocial scientists, and more.
A lot of our work, we'verelied on these surveys.
(25:08):
And in the case of someof this work, perspective
transformation,we're looking at,
did people challenge theirprior presuppositions
that they came in with?
And the learningsciences theory on this,
it suggests that thereare multiple stages
of prospective transformation,and essentially, we're
looking to see if peopletransform their perspectives
(25:29):
from when they began.
And I mean, there area lot of instruments.
We've done work ondiscrimination in schools.
So my former PhD studentwho graduated a year ago,
Danielle Olsen, thesis was on VRsimulation of racist experiences
in education.
And that used a model calledracial and ethnic socialization
(25:54):
model.
And in that case,it both assesses
users to see whatcategory of socialization
that they've beenraised through, that is,
through parents, through media,through peers, and so forth.
There were fourcategories in that case.
One is called a colorblindcategory, where people say,
(26:14):
I don't see race.
It doesn't exist.
We shouldn't talk about it.
There's another category,which is distrust
of other people,which fortunately is
the one that you see the least.
But it is out there.
I mean, distrust of peoplefrom different groups.
You have prepared for bias.
Sometimes people thinkabout it these days
as related to the talkas an example of it.
(26:35):
That's the talk that people inunderrepresented or oppressed
groups in society might saygive their children when
having to potentially deal with,say, biased authority figures,
and say, you mightexperience this bias.
Here's how you might want tocomport yourself, and so forth.
And then the final one ispride and legacy reinforcement,
(26:56):
which is a social acceptance andpride of not only your group,
but the groups ofothers, and looking
at the positive characteristicsthat might exist
within those different groups.
And so we essentiallywere able to,
through this interactiveexperience, where
the user interactsthrough selecting gestures
for the character, assess howthey were socialized to think
(27:18):
about race from the experience.
And then it can alsobe an intervention
to help them then reflect on andhelp with a kind of perspective
change based upon themthinking through the issues
as they go throughthe simulation.
Yeah, so I have one lastquestion from my side,
and it has to do a littlebit with the metaverse.
(27:41):
As you know, atMIT, a lot of people
have been talking aboutmetaverse and the creation of, I
would call it, augmented realityto navigate the complex world
of the internet, and so forth.
Do you think that this isgoing to help us understand
the world a littlebit better, or is it
going to isolate us evenmore from human beings
(28:04):
and the rest of the world?
And I mentionedthis as an example,
where recently, we visited Peruand we went to Machu Picchu.
And it's a trek,and it's actually
complicated to getto Machu Picchu.
And the question is, I couldhave done a metaverse on this
and got some ideaof the whole place.
Is that a good alternative?
How is that affecting us ashuman beings focusing so much
(28:27):
more about thisaugmented reality?
Actually, I willclarify, just to say,
because I don't think it'sjust augmented reality.
I think it's somethinglike virtual worlds
with social media, affordances,and networked computing
all coming together.
And you could think aboutthe metaverse as something
like this.
And I think that, like a lot ofthings, it depends on the use.
(28:55):
My PhD advisor, JosephGauguin, was staunchly
against what's known astechnological determinism,
the idea that any technologyis a unilateral force that
will shape thefuture of society,
for better or for worse.
Any technological deterministstatement is false,
(29:17):
is what he used to say.
And people would argue,they'd say, but certainly,
the cell phone,that's changed life.
Or people might say, nowadays,certainly, the metaverse,
that has changed society.
And in that case, therewould be missing the key word
in this, which is theunilateral side of it.
And so the alternativeway that I think about it,
(29:39):
I think it's in line withthinkers like sociologists Bruno
Latour and others,is it's a ecology.
There's a network ofdifferent forces at work
that includes people's, say,everyday physical relationships,
the relationships online,the technology itself,
the documentationof the technology,
(30:00):
the way it's reported onin the media, and so forth.
You could say, the effectscome from that whole network
of forces.
And I think that's what makesit a complex question to answer.
But if I wanted just to givea personal response, just
my own feeling about it,is that, one, I think,
(30:20):
if the experiences people haveultimately pushed people back
to the physical world andphysical world relationships,
I think there's somethingabout not making
it a surrogate for physicalworld relationships
and experiences,but something that
loops you back towards thosekind of experiences and needs.
(30:42):
If it just absorbs you intosomething which is simulated,
there's just so many dangers.
It can give the illusion ofthe experiences you would have
in the physical worldwith all of that nuance,
but really, it'shighly regulated,
created by a system ofindustry, and so forth.
So the nutshellversion is I would
say that there's quitea danger that there,
(31:04):
although, if designed incareful and sensitive ways,
there also are profoundlypositive opportunities that
are available.
And hopefully, manyof the designers
are cognitive of thisissue when they design
aspects of the metaverse.
I hope, increasingly so.
I mean, it just dependson the designers.
(31:26):
But they're gettingto be some practices.
I guess, you could say it'slike in other areas of software
engineering thatthere are many cases
where people only thinkabout individual users
or only think about, let's say,the uptake of the platform,
and so forth.
But there are strands ofhuman-computer interaction
and softwareengineering, where people
(31:47):
think about social impact, buildthat into the process of design,
work with variousstakeholder groups,
involved the stakeholdergroups in the design process
and evaluationprocess, and so forth.
And I think it's similar.
We begin to see some worksthat are developed hand-in-hand
with community members.
Just now, I'm on a grantwith researchers in Canada,
(32:09):
and also, I mean,just internationally
on indigenous cultures andartificial intelligence,
and just thinkingabout the impacts
on different global indigenousgroups, and so forth.
And so I think that there are anincreasing number of initiatives
that take this kindof work into account.
I think that's maybe one ofthe hallmarks of the Institute
for Data Systemsand Society at MIT,
(32:30):
is just the factthat people are both
looking at those socialfactors, as well as
the information and dataeconomy side of the work.
If you use that as aparallel, you could ask,
how many departments aredoing something similar that
take into account thetechnical and the social,
(32:50):
and how many are focusedprimarily on just the end
use in a more technical sense?
And that might be parallelto other types of software,
like entertainmentand metaverse, too.
Thank you.
So Fox, this is totally outthere and your personal opinion
on this, but in terms ofwhat is the future of this,
(33:11):
is there a world in whicheveryone walking around
has virtual realitygoggles on and everyone
is living in this simulatedworld that's not physical, or is
this just the stuff of movies?
I think if we extrapolate fromwhere we are now, as everybody
(33:34):
who has even a few yearsago, we call it a smartphone,
but now it feels almostredundant to say smartphone
because everybodyjust has a phone that
is of this sort carrying arounda computer in your pocket.
And so I think the idea thatpeople will be networked,
that people have virtualsurrogates for themselves,
(33:57):
different worlds toexplore, and ways
to engage withcomputational media, that
certainly seems to be the case.
Whether that lookslike something
like a headset orglasses, and so on,
I think that wouldremain to be seen.
(34:17):
I think it will happen ina seamless kind of way,
like the integration ofthe phone and the camera.
At first, people thoughtthese are just two devices.
Why would they haveto be paired together?
And incrementally, peoplebegan to be used more and more
(34:39):
to having different formsof mediation of life
that you're carryingaround with you.
But I think there's a problemwith the headset, in the idea
that, again, blockingthe physical world
and then replacing itwith something else,
but overlay in ways thatseem banal initially.
(35:00):
But the same way that we scana grocery store card and that
connects us to a nationalor global network of data,
and then that datatransforms our experience
of the store because it guideswhich products are going
to be placed therenext, and so forth,
it's very easy to imaginepicking up your phone
(35:22):
and now you see informationabout those vegetables
or whatever is in front of you.
And then just incrementally,a bit more and more.
Maybe advertisingcreeps in, and so forth.
So I think it willhappen more like that,
not a spectacle, in which,within a few years everybody
is immersed in thisglobal illusion.
(35:45):
But we'll become less andless sensitized to the way
that data is overlaid onour daily experiences.
And it might look not somuch like being immersed
in some exciting virtual world,like the Holodeck in Star
Trek, The NextGeneration, but it
(36:06):
will be more likewhat we experience
of going online now withour phones in the store
or walking around, where you'rejust a little less connected
to where you are.
You have access to a lot ofdifferent types of information.
You want to know a little bitabout that musical instrument
you're going to buy.
So let me look it up.
But now, instead, youjust see it laid on top.
(36:28):
I could imagine things justgoing in that direction
if I were to extrapolate ina fanciful way just based
on where we are now.
Thank you so muchfor doing this.
We know how busy you are, so wereally can't thank you enough.
Thanks, Fox.
Every time I listen toyou, I learn something more
about that word.
Thanks.
(36:48):
It was absolutely my pleasure.
Thank you for listening to thismonth's episode of Data Nation.
You can get more informationand listen to previous episodes
at our website,idss.mit.edu or follow us
on Twitter andInstagram @MITIDSS,
@LibertyVittert,and @MuntherDahleh.
If you like thispodcast, please don't
(37:08):
forget to leave us areview on Spotify, Apple,
or wherever youget your podcasts.
Thanks for listeningto Data Nation
from the MIT Institute ofData Systems and Society.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy And Charlamagne Tha God!
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.