October 10, 2025 • 31 mins
As a blind video gamer, the options for the kinds of games you can play are limited. Not only are accessibility features not a standard practice in game development, the tools available right now only go so far. Exploration – a fundamental part of the fun in gaming, especially in an open world game – is often not really possible for blind players in today’s games. Researchers at Columbia University are working to change that. Dexter talks to Brian Smith and Vishnu Nair, who are developing tools for blind and low vision gamers to be able to play exploration-based games that could one day even be applied to a game like Zelda Breath of the Wild.
Got something you’re curious about? Hit us up killswitch@kaleidoscope.nyc, or @killswitchpod, or @dexdigi on IG or Bluesky.
Read + Watch:
See omnystudio.com/listener for privacy information.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:10):
Here we go.
Speaker 2 (00:13):
This is a racing game called for it's a motorsport.
Aside from the sound effects from the driving, you might
also notice those beaps. Brandon Cole is completely blind, and
this is how he plays the game.
Speaker 1 (00:31):
Going to the last turn.
Speaker 2 (00:34):
Decide that the beeps are on, and also the pitch
and the intensity gives an information on everything from where
he is on the track, whether or not he's too
close to the wall, to how sharp the next turn is.
Speaker 1 (00:45):
I'm so glad I was able to do that turn.
That's actually a very difficult turn.
Speaker 3 (00:48):
Oh that's why thirty year actually every count fifteen point
two six five seconds.
Speaker 2 (00:54):
Brandon Cole passed away last year, but he wasn't just
a gamer. He was also a consultant on this game.
And this is him demonstrating the game on his YouTube channel.
Speaker 1 (01:04):
But just rewind that rewinding. I want to show you this.
Speaker 4 (01:10):
Rewind done.
Speaker 3 (01:11):
I want to show you that because the blind drivings,
sounds play while you're rewinding.
Speaker 2 (01:14):
If we're talking about racing games, there is a before
Fortsa and an after Fortza. Before this game came out,
it wasn't possible really for a blind person to play
racing games.
Speaker 4 (01:25):
But now they can.
Speaker 3 (01:26):
Hey, other blind person, I offer you control. I offer
you the opportunity to get into a vehicle of your choice,
pick the track of your choice, and once you're on
that track, the ability to drive it yourself, to take
those turns at high speeds yourself.
Speaker 1 (01:45):
This is your road.
Speaker 4 (01:46):
Now, Welcome to Forza.
Speaker 2 (01:49):
Forts of the game was put out by Microsoft, but
the technology that made those blind accessibility features possible actually
came from a small research lab in New York led
by a professor named brad Ian Smith.
Speaker 4 (02:01):
When we first.
Speaker 2 (02:02):
Spoke, it was early mid twenty twenty, I think, and
you were telling me about this experimental thing that your
lab had come up with where blind people could play
a racing game, and you were very confident you were
going to get in the game. I don't know if
I was as confident.
Speaker 5 (02:23):
Because it sounded great, but I maybe I was just
pessimistic about the good front. But yeah, but you know,
I mean I think it was more than like, Okay,
this sounds amazing, but the step is for a game
company needs to adopt this thing.
Speaker 4 (02:38):
But it happened.
Speaker 2 (02:40):
Brian is an assistant professor of computer science at Columbia
University and he's the head of a lab called Seal
that helped make the first blind accessible Triple A racing game.
Speaker 6 (02:50):
What we're most known for is our work in assiste
of technology, in particular technology for people who are blind
or low vision.
Speaker 2 (02:56):
Their work goes beyond Fortza, and it goes beyond video games.
Speaker 6 (03:01):
Video games are but one example of a visual experience.
By figuring out how we can enable that fundamental ability
to perceive an image or a screen, we can actually
unlock not only access to video games, but all sorts
of visual experiences in daily life.
Speaker 2 (03:27):
I'm afraid from Kaleidoscope and iHeart podcasts.
Speaker 4 (03:34):
This is kill Switch. I'm Dexor Thomas.
Speaker 1 (03:40):
I'm sall right, good guy.
Speaker 2 (04:15):
How did you become interested in how blind people play
video games?
Speaker 1 (04:20):
It all started with a YouTube video that I actually
encountered while I was a PhD student. It was a
video of a blind person by the name of Terry Garrett.
He was playing a game.
Speaker 6 (04:33):
Called The Legend of Zelda Ocher Enough Time, which is
a classic Zelda adventure game that I remember playing as
a kid as well. Yeah, the interesting thing about Terry
Garrett was that he wasn't blind his entire life, like
he actually grew up as a little kid playing that
game and then later lost his vision, and as soon
as he lost his vision, he almost got shut out
of video games completely.
Speaker 1 (04:55):
For years, the only things he could play was some
games that he could find online would were called audio games,
which were games specifically designed for people who are blind
that are audio only.
Speaker 6 (05:06):
You know, in a lot of ways, they're very distilled
experiences compared to what sited players would be able to play.
Speaker 1 (05:12):
So one day Terry decided to pick up Zelda again
and say, though, I want to play this game.
Speaker 6 (05:17):
And I'm going to do it no matter what. And
it took a whole bunch of trial and error. He'd
do a whole bunch of like hacks that he figured out. So,
for example, he would walk around a room and whack
the walls with his sword again and again and again
until he heard a different kind of clinking noise which
told him like, oh, that's the door instead of the
rest of the wall, and so that's how I might
get into this room.
Speaker 1 (05:38):
There he'd memorize a whole bunch of details, and.
Speaker 6 (05:42):
That really landed for me because like it showed me,
here's something that i'd take for granted. These video game experiences, right,
and there are a lot of people who go through
so much effort just to have some semblance of that experience.
Speaker 1 (05:56):
Like I was you know, I was floored.
Speaker 6 (05:57):
I was amazed, But at the same time I felt pain.
I felt like, wow, like this is not a good
state of affairs right now. We've got to do better
than this.
Speaker 1 (06:09):
This isn't right.
Speaker 6 (06:10):
And that motivated me as like, Okay, here's a place
where just rethinking.
Speaker 1 (06:15):
How we're designing these games, these computer systems.
Speaker 6 (06:18):
Really can really enable people to have new types of
meaningful experiences.
Speaker 2 (06:23):
Yeah, you know what, We've gotten to the point where
we've legitimately got cars that drive themselves. And when there's
a kid who wants to play a videome, or an
adult for that matter, who wants to play a video
game who can't see the screen, we sort of shrug
our shoulders, I think, collectively, and say, you're asking a
little bit too much, buddy.
Speaker 1 (06:45):
Yeah, that's right.
Speaker 2 (06:46):
And you said no, you said no, I don't think
you are that's right. There are more games out now
that have accessibility features for blind and low vision players, like,
for example, there's SPORTSA and there's also the last of
Us too, which we mentioned in the last episode. But
that's a handful of games out of well a lot
of video games, and it's probably not fair to compare
(07:06):
the big blockbuster games to the smaller independent games, but
they do have something in common. Most games, especially the
blockbuster ones, the Triple A games what we call them,
are a lot further behind than they could be in
terms of blind accessibility and accessibility in general.
Speaker 6 (07:22):
Most mainstream games, the kinds of games you'd buy on
Steam or Xbox, PlayStation, Nintendo, are barely accessible to people
with disabilities. That includes people who are blind to low vision,
people who are deaf or hard of hearing, people with
motor impairments, people who have a hard time like mashing
buttons or pressing too many buttons too quickly.
Speaker 1 (07:44):
In sequence, that's the current state of things.
Speaker 2 (07:47):
Game companies are starting to take accessibility more seriously. Some
of this stuff seems kind of obvious, like the ability
to make text larger, or the ability to switch to
a color blind mode, or an option to read out
the text on the screen. But in order for a
blind or a low vision player to be able to
truly experience a video game, and anything close to the
way that say, I experience it, that game needs to
(08:10):
be built with that player's experience in mind from the start.
But unfortunately, the industry standard right now is to essentially
dumb down the game for that player.
Speaker 6 (08:20):
Oftentimes, the basic approach to accessibility is just to make
the game a little bit easier that actually take away
agency and choice from the player. So, for example, they're
video games that have accessibility features that allow players to
skip different sections or find items faster, but they do
(08:41):
so in a way that almost plays the game for
the player, like, oh, you can't really see the path ahead,
you can't really see the image.
Speaker 1 (08:49):
Well, we'll press this button and I'll just warp you there.
Speaker 6 (08:53):
And that form of accessibility does not offer players the
same type of experience, and that's the more challenging type
of accessibility problem.
Speaker 2 (09:02):
To solve this type of accessibility of making the game
easier to play and taking away some of the agency
of the player. You can even see that in the
Last of Us, and this is the game that's arguably
gotten the most positive attention for their accessibility features.
Speaker 7 (09:17):
What The Last of Us does is you can hold
a button to automatically move your character to the next
waypoint to the next object of interest.
Speaker 2 (09:26):
Vishnudnair is a research engineer and he was a PhD
student in Brian's lab, which started as just the two
of them working on video game accessibility technology.
Speaker 7 (09:35):
A lot of the players that we talked to called
it the golden path. That's all it does. We refer
to it as guidance. These are guidance based systems and
that's all they do. It's like you're taking their hand
and dragging them wherever, and that's not great for independence.
And it's that independence and autonomy, that ability to make
(09:57):
decisions on what I want to do, Which do I
want to turn that corner and maybe find a treasure
chest or if I turn that corner, maybe I'll encounter
a very powerful enemy and die. You know, making those
decisions and facing those consequences is like the most important
part of having fun, and being guided doesn't allow you
to have those experiences.
Speaker 2 (10:20):
This Golden Path will basically walk you from point to
point in the game, basically just getting into the next
spot to keep the story moving. It is well intentioned,
but it doesn't give blind players the same experience as
a sided player. This is gonna sound weird, but it
turns out that letting a blind player drive a race
car is relatively pretty easy. What's hard is letting that
(10:41):
blind player get out the race car and just walk around.
Speaker 4 (10:45):
But Brian and Vishnu think they got to figure it out.
Speaker 2 (10:48):
That's after the break. So let's go back to Terry Garrett.
That's the blind gamer who be Zelda sixty four. The
way he navigated the game was by going up to
a wall and hitting it with his sword, and based
on the sounds that he heard as he went down
(11:08):
the wall hitting it, he could tell if that was
just part of a normal wall, or maybe a door
or a secret passageway. This was a really clever workaround
for a game that was not meant to be blind accessible. Nowadays,
we have games that are developed with blind accessibility in mind.
They incorporate what Vishnu might call that golden path. So
what Terry was having to do was definitely tedious, and
(11:31):
a golden path approach is way more efficient. That sounds great,
but let's step back for a second. Does more efficiency
actually equal fun?
Speaker 6 (11:43):
You can imagine a game like Zelda right where we
have the main character Link and around Link might be
a ladder to a second floor, and then a door
into another room, and then a treasure chest or some
bushes that you can cut for rupees the game's currency,
et cetera.
Speaker 1 (12:00):
Objects around link at any given time, and.
Speaker 6 (12:03):
As a cited person, we can just move the camera around,
look around, see what those things are, figure out which
way we want to go, what we want to do next.
Speaker 1 (12:11):
But as a blind person, that experience is very different.
Speaker 6 (12:15):
Even with some of the cutting edge accessibility features that
are out there in games right now, the way that
that experience would play out is the blind player might
like press a button to just open a menu that
reads out the objects around them, so it might say door, ladder,
treasure chest, bush, whatever, like that's it. It's just a
(12:36):
linear menu, and oftentimes that menu kind of waters down
the experience of looking around and exploring. So, for example,
if the whole point is to find like a small
key hidden somewhere in some nook or cranny that might
unlock a door on another side of the dungeon, if
you pulled up the menu and the menu just said
small key, it kind of spoil the.
Speaker 2 (12:57):
Surprise, right, yeah, yeah, you not playing a game. It's
like you're on a guided tour.
Speaker 6 (13:03):
Exactly exactly, and so that experience of like exploration and
discovery is missing from those current accessibility tools.
Speaker 2 (13:14):
But how do you define that experience? So this is
gonna sound totally unhinged, but hear me out. There's this
YouTuber who has this series where he goes to what
he calls unremarkable and odd places in Zelda, like places
where you just go and there's a couple of rocks
near a river or something. Has nothing to do with
the progressing of the game. There's no mission critical item there.
(13:34):
It's just a river and some rocks. And one of
his videos has three hundred thousand views, and people in
the comments are saying, yeah, I remember that spot.
Speaker 4 (13:42):
I love that spot. I like hanging out at that spot.
Speaker 2 (13:45):
So how do we give blind people the closest thing
to getting a game and exploring, not accomplishing things, not
checking stuff off of a checklist, but playing the game
on your own terms. Brian in his last are taking
this question very seriously, and they published two papers on
applications that facilitate precisely this ability. One application is called
(14:09):
navstick and the other it's called Surveyor.
Speaker 6 (14:13):
So both Navstick and Surveyor are about making three D
adventure games more meaningfully accessible to blind players. So what
vish news work was really about was what's important to
the experience of exploration and discovery? When cited people are
playing Zelda? What do they do at one moment in
(14:34):
time the next moment of time, How are they having fun?
What choices are they making at any moment in time,
and like, what information are they exploring to figure that out?
And now, how can we make that process of exploration
and discovery more meaningfully accessible?
Speaker 2 (14:51):
The first step was figuring out how a blind player
could explore their immediate surroundings so that they could understand
what choices they had without just doing something for them
or telling them where the thing that they were looking
for was.
Speaker 7 (15:03):
We were like, okay, a person kind of rotates their
head all around them to look around themselves and get
a map of what's imediately around them. How do we
give that ability to someone with blindness? And so that
was where navstick came along, where it was really simple.
We just repurposed the right thumbstick on the standard game controller,
(15:24):
and all you have to do is just point the
right thumbstick in some direction and whatever lies in that direction.
The game's going to tell you that was it.
Speaker 2 (15:33):
So I tried navstick and it's really intuitive. Basically, you
move your right thumbstick around and whatever you pointed at,
a voice plays and tells you.
Speaker 4 (15:42):
What it is. Clean, Red Dirk Chocolate, Chocolate, Chippickies Better.
Speaker 2 (15:47):
This is a clip of Vishnu demonstrating navstick in a
virtual grocery store setting. You can hear the sound of
the voice moving around depending on where the item is
in relationship to you.
Speaker 7 (15:57):
Christians Coffee Grounds, Dirk Chuck. We actually created an entire
video game to test this, and everyone loved it. What
was amazing was that, you know, we had we put
in all these levels. All these levels had like different
trees and enemies that they had to shoot using a
laser gun. Every participant completed every level of that game,
(16:18):
no matter how complex it got. And what we found
was that everyone crapped at different strategies to get through
those levels. Maybe they went this way to the left
and defeated a few enemies to get to the finish line,
or maybe they just barrel straight through. They were able
to make those decisions and everyone said that, oh, this
is like just so much more fun than using like
(16:38):
a menu for instance, and just being told where to go,
I was able to like actually feel like I was
playing the game.
Speaker 2 (16:46):
This is a pretty simple application, but it's very powerful.
So I've tried doing the thing where you close your
eyes and make a mental map of your surroundings and listen, man,
I suck at it. This is something that blind people
are usually very good at because because they have to be,
but it can even for them get kind of taxing.
Navstick means that you don't have to memorize a brand
(17:07):
new area when you first go to it. You can
check where you're at right then. But for a full
three D game, especially a big game, napstick is just
one part of that equation.
Speaker 7 (17:18):
The question beyond navstick is how do we allow a
blind user to experience a larger area. You have this
entire dungeon that you have to explore. How do you
handle that? How do you break away from that guidance
paradigm where it's like you're leading a person by the
hand and leading them to the treasure chest. In game
(17:41):
design theory, discovery is one of the main elements of
video games that are important for a good experience, and
so how do we facilitate that, and so we create
this entire system Surveyor where they would use navstick, but
in addition to navstick, we help keep a law of
where they've been. Basically, if you were to take an
(18:04):
overhead view of a dungeon and you color in the
places where the person has been, the point is you
want to bring a user to the edge of the
area those borders between where they have explored and where
they haven't explored. That's what we did with Surveyor, where
effectively it would say, okay, about fifty feet away, there
(18:28):
is an unexplored area, and if you select this, you'll
be guided there. And once you reach that unknown the
guidance cuts off and you're left using knapstick to look
around that area.
Speaker 2 (18:42):
Oh so, it's kind of like fast traveling a game. Exactly,
I've already explored this part. Don't make me run across
a field for twenty minutes. I've already been here, I've
completed this area of the game. Let me get back
to the town. Cool, and it takes you to the
edge of where you've been, drops you off there and says, okay, figure.
Speaker 7 (19:00):
It out exactly exactly, and that's what promotes that sense
of autonomy. That's what promotes that sense of discovery. What
we're trying to do is, Okay, we don't need to
confine the player anymore, right if we can just open
it up, just take something like Tiers of the Kingdom
and put a player with surveyor in it. Can they
(19:21):
just explore that area, these cities, these little dungeons and
all that.
Speaker 2 (19:26):
Zelda Breath of the Wild and its sequel, Tiers of
the Kingdom are two of the biggest open world games
that are out right now. They're really popular, and it's
not just because they're good, but because they're really approachable.
I have friends who don't play a whole lot of
video games, but you know, they'll turn on the switch,
They'll jump in Zelda, They'll go run around in a forest,
not even fighting enemies, just having fun. Zelda Tears of
(19:49):
the Kingdom. You think you could put this onto a
Zelda Breath of the Wild type game and people be
able to play it.
Speaker 7 (19:57):
Perhaps, I mean, they are obviously going to be some
design decisions that to be made, But in my defense presentation,
I literally use Tears of the Kingdom as an example.
Speaker 2 (20:06):
Again, blind people can play some video games now and
feel that sense of satisfaction of accomplishing a task, like
finally beating that last boss. But that's not all a
video game is. I mean the freedom to just run around, explore,
hang out, that isn't there for a blind player. But
now with these tools that Vishio and Brian have developed,
a blind players, say like Ross Minor from last episode,
(20:28):
could probably do precisely that, just go wander around without
assistance or special modifications. That potentially opens things up a lot.
Speaker 7 (20:38):
Yeah, that opens a lot of social opportunities as well.
If we can open up the same experiences to blind
players as side of players have, then we can foster
better connections at the end of the day.
Speaker 2 (20:53):
I mean that totally makes sense. I mean, you know,
if I found out somebody played Earthbound, we're immediately friends immediately,
you know what I mean.
Speaker 4 (21:00):
Or same thing with really.
Speaker 2 (21:03):
Anything, music, movies, books, whatever, You've experienced, the same thing,
you read, the same stuff, you listen to the same music.
Obviously video games figure into that pretty heavily too, right,
So you may think this all sounds really exciting, but
you're wondering when this stuff's gonna come out, or you
might think that this is an awful lot of effort,
(21:23):
and you're wondering why these brilliant researchers are wasting their
time on video games. Well, I got two answers for you.
That's after the break. The tools that Brian and Vishnu
(21:45):
were talking about are pretty incredible, but they're also really
intuitive and simple. It kind of makes you wonder why
stuff like this is so rare. Well, A big part
of the problem is that game companies are really siloed.
They don't talk to each other, and they definitely aren't
willing to share their secrets with each other. I've had
numerous people tell me who develop games, either at a
(22:07):
big company or even especially if they're an independent developer,
They'll say, look, I would love to make my game accessible.
It would take me six months. I'd have to stop
everything I'm doing and I'd have to figure it out somehow.
And realistically speaking, that six months and I actually can't
afford to do that.
Speaker 6 (22:25):
Yeah, And I think that the reason behind that is,
like every studio has to build these things from scratch.
Speaker 1 (22:33):
There aren't enough of.
Speaker 6 (22:35):
These like accessibility widgets that are available like just kind
of as a package, ready to go that you can
plug in.
Speaker 2 (22:42):
So that's exactly what Brian and Vishnu had been working
on within the lab.
Speaker 6 (22:46):
We're actually creating a plug in for Unreal Engine to
make it easier for any game built on Unreal Engine
to adopt napstick, and then we'll later add survey or
functionality to that.
Speaker 2 (23:00):
Reel is a game engine that a lot of Triple
A games, that's those big blockbuster games are built on.
Speaker 4 (23:05):
So an engine is.
Speaker 2 (23:07):
Kind of like the base program that a lot of
game developers build their game on top of. So, for example,
if you're making an adventure game where you walk around,
if you had to program in the concept of gravity
or the concept of a ground that you walk on,
you'd never get anything done. So you get an engine
like Unreal that has a lot of the basic stuff
built in, and if you need additional features, you get
(23:29):
a plug in. So there are plugins that'll help you
make realistic trees or anime an ocean or a river,
or make realistic sound effects. And what Brian is hoping
to do is to make a plug in that any
game developer who's using Unreal could just bring into their
engine and add the navstick or the surveyor features.
Speaker 1 (23:48):
So we are going to put as a law. We're
going to publish it on Unreal Engine marketplace so.
Speaker 6 (23:52):
That developers can see the possibilities and see exactly like
how to implement and incorporate these things into the game.
Speaker 4 (24:00):
How much would this cost for a developer.
Speaker 1 (24:02):
It would be minimal. I don't expect it to cost
more than say, like one hundred bucks or something like that.
Speaker 4 (24:08):
Okay.
Speaker 2 (24:09):
Yeah, And the ability to throw that in and potentially
have some more people by your game who would never
have picked it up is kind of cool.
Speaker 6 (24:17):
Absolutely, Yeah. And I would love to, you know, for like,
especially like independent developers and things like that. You know,
I'd love to have a program where they can they
get incorporated into their game without having to pay. This
is something I'm still working with the university on.
Speaker 2 (24:34):
The hope is that these plugins make it so obviously
easy to add in accessibility features that developers see it
and they say, you know what, let me just download
this plug in and just do it, and they'll be
able to get started pretty soon.
Speaker 6 (24:46):
We're actually hoping to launch that by the end of
this calendar year twenty twenty five.
Speaker 4 (24:51):
Wow.
Speaker 2 (24:52):
I really do hope that game developers check this out
because I'd love to see what kind of games come
out in the next few years that use this technology,
because this isn't just a niche feature. It really can't
open up games for an entire community of people who've
been shut out of video games, and that can open
up new ways for everyone to connect. I remember early Quarantine,
(25:13):
my niece was in a different state, and when we
wanted to communicate, she didn't call me. She said, Yo,
let's get on Fortnite, and that's how we talked. We
got in Fortnite and we ran around and build stuff,
and you know, she showed me some weird little structure
that she'd built and her friends are running around in there,
(25:34):
and like, that's how we talked. And so video games
now are, for better or worse, where people spend time
with each other.
Speaker 1 (25:44):
If they're immersive, right, it's a form of immersion.
Speaker 6 (25:47):
And so I think that, like you know, any tool
that can can enable access to video games could also
enable access to a whole host of immersive experiences as well,
like being able to like tour apartments, for example, without
having to visit that apartment or two or a different city.
Speaker 7 (26:04):
There's potential to bring nastick something like nastick into the
real world as well. Really yeah, navigation through the real
world using you know, you have a blind person outside
or in a building, you know, pointing in some direction,
maybe with your camera or something else, and having it
say what what's around you.
Speaker 2 (26:26):
This is a pretty interesting idea because it could be
an augmented version of what blind and low vision users
have found in the meta smart glasses. If you haven't
seen our episode on that, you can check that out.
But instead of AI as most people think of it,
this would be layering a video game kind of on
top of reality.
Speaker 1 (26:47):
An example that I like to give is New York's
Pen Station.
Speaker 6 (26:51):
It's the busiest train station in the world, and it's
massively complex to someone who's never been there before, with
multiple railroads and subway lines and you know, et cetera.
Speaker 1 (27:01):
It's huge.
Speaker 6 (27:03):
And imagine being able to like swipe by like tapping
the side of an earbut and exploring kind of circularly
and being able to hear spatial audio of like what's
around you at Penn Station.
Speaker 1 (27:16):
Oh, the ticket office is here, that's track twelve, that's
track thirteen.
Speaker 6 (27:20):
Here's the McDonald's, or here's the shake shack, right we
have this physical world, we just have to create a
virtual representation of it first.
Speaker 1 (27:28):
Let's say like a video game version, a digital version.
Speaker 6 (27:31):
Oh so that like now, when we're in the physical world,
if the system knows where you are, it's the same
as napstick. You're exploring that environment, but you just happen
to be physically inside the place as well.
Speaker 2 (27:43):
The video game stuff that I'm messing with and that
in the future gamers will be able to use ends
up kind of being like a trial run for using
this in real life at a train station or a
grocery store.
Speaker 6 (27:56):
Yeah, that's a great way of thinking about it. Oftentimes
we think of the games that we prototype as being
almost like testing grounds, Right, that's our little our peatrie
dish really to test our systems. But it's just a
fake version of what might be a real world environment
as well.
Speaker 2 (28:18):
If you told someone in twenty fifteen that they wouldn't
be able to walk to their mailbox without being recorded,
they would think that you were describing as science fiction dystopia.
If you say that now in twenty twenty five, they'd say, well, yeah,
you're just describing the basic function of my neighbor's ring
camera we now expect technology to keep expanding, so what
if we started expecting accessibility to advance also. I mean
(28:41):
five years ago, I thought Brian was nuts for thinking
that Ross Minor would be able to play a racing game.
Speaker 4 (28:47):
And then this year I.
Speaker 2 (28:48):
Walk into Ross's apartment and He's got a driving wheel
on his desk.
Speaker 4 (28:51):
To play Fortza.
Speaker 2 (28:52):
But this stuff doesn't just happen. It's the industry, it's
the universities, and it's the gamers. I'm not saying you
need to write a strongly worded email to Nintendo, but
you do have influence over things, and they do affect everyone.
I mean, look, I personally do not need Knavstick and
I don't need Survey Air, but I am more likely
(29:12):
to buy a game if I know that it means
that I might get to play co op with Ross.
The stuff that people like Brian and Vishnu and Ross
build is gonna make games more interesting for everyone. I mean,
who knows, Maybe someone listening to this right now, maybe
you could end up contributing something to that process. And
(29:34):
thank you for listening to kill Switch. This was a
fun one to do, and hey, if you got any
ideas for the next one, you can hit us on
email at kill Switch at Kaleidoscope dot NYC, or we're
on Instagram at kill Switch Pod. And if you listen
all the way through our first double episode, hopefully that
means you liked it. So you know, get that phone
out and please leave us a review. I know people
(29:56):
say it all the time, but it really helps other
people find the show, which he'll us keep doing our thing.
Killswitch is hosted by Me Dexter Thomas. It's produced by
Sena Ozaki, Darluck Potts, and Julia Nutter. A theme song
is by me and Kyle Murdoch and Kyle also mixed
the show from Kaleidoscope. Our executive producers are Ozma Lashin,
Mangesh Hatigador, and Kate Osmore from iHeart Our executive producers
(30:21):
are Katrina Norvil and Nikki Etour. Oh one more thing.
Oh this is for all my game developers out there
who are listening to this. You might have been a
little disappointed to hear that they're making these plugins for
Unreal Engine, because look, Unreal is great. A lot of
blockbuster games are built on it, but it's just that
it's kind of for the big time games. You know,
why would you leave the independent game developers out, the
ones who you know, use different software, cheaper development software
(30:43):
like say Unity. Well, I asked Brian about that.
Speaker 6 (30:47):
I wanted to follow the model that happened with Fortsa,
where like a major developer had this feature incorporated into
their game that got noticed by the whole industry. That's
an important and first domino to fall. And so we
do want to make a Unity equivalent of the plug
in as well. But we chose Unreal first for that reason,
(31:08):
so that we could work more closely with some of
the bigger developers, work through all the kinks with their
larger teams they would have more resources, and then figure
out how to tailor both that Unreal Engine plug in
and how to do things right on Unity for the
smaller indie developers the first time.
Speaker 2 (31:28):
So there's your answer, and you know I get the
approach anyway, Catch on the next one, good guy,
Here we go.
Speaker 2 (00:13):
This is a racing game called for it's a motorsport.
Aside from the sound effects from the driving, you might
also notice those beaps. Brandon Cole is completely blind, and
this is how he plays the game.
Speaker 1 (00:31):
Going to the last turn.
Speaker 2 (00:34):
Decide that the beeps are on, and also the pitch
and the intensity gives an information on everything from where
he is on the track, whether or not he's too
close to the wall, to how sharp the next turn is.
Speaker 1 (00:45):
I'm so glad I was able to do that turn.
That's actually a very difficult turn.
Speaker 3 (00:48):
Oh that's why thirty year actually every count fifteen point
two six five seconds.
Speaker 2 (00:54):
Brandon Cole passed away last year, but he wasn't just
a gamer. He was also a consultant on this game.
And this is him demonstrating the game on his YouTube channel.
Speaker 1 (01:04):
But just rewind that rewinding. I want to show you this.
Speaker 4 (01:10):
Rewind done.
Speaker 3 (01:11):
I want to show you that because the blind drivings,
sounds play while you're rewinding.
Speaker 2 (01:14):
If we're talking about racing games, there is a before
Fortsa and an after Fortza. Before this game came out,
it wasn't possible really for a blind person to play
racing games.
Speaker 4 (01:25):
But now they can.
Speaker 3 (01:26):
Hey, other blind person, I offer you control. I offer
you the opportunity to get into a vehicle of your choice,
pick the track of your choice, and once you're on
that track, the ability to drive it yourself, to take
those turns at high speeds yourself.
Speaker 1 (01:45):
This is your road.
Speaker 4 (01:46):
Now, Welcome to Forza.
Speaker 2 (01:49):
Forts of the game was put out by Microsoft, but
the technology that made those blind accessibility features possible actually
came from a small research lab in New York led
by a professor named brad Ian Smith.
Speaker 4 (02:01):
When we first.
Speaker 2 (02:02):
Spoke, it was early mid twenty twenty, I think, and
you were telling me about this experimental thing that your
lab had come up with where blind people could play
a racing game, and you were very confident you were
going to get in the game. I don't know if
I was as confident.
Speaker 5 (02:23):
Because it sounded great, but I maybe I was just
pessimistic about the good front. But yeah, but you know,
I mean I think it was more than like, Okay,
this sounds amazing, but the step is for a game
company needs to adopt this thing.
Speaker 4 (02:38):
But it happened.
Speaker 2 (02:40):
Brian is an assistant professor of computer science at Columbia
University and he's the head of a lab called Seal
that helped make the first blind accessible Triple A racing game.
Speaker 6 (02:50):
What we're most known for is our work in assiste
of technology, in particular technology for people who are blind
or low vision.
Speaker 2 (02:56):
Their work goes beyond Fortza, and it goes beyond video games.
Speaker 6 (03:01):
Video games are but one example of a visual experience.
By figuring out how we can enable that fundamental ability
to perceive an image or a screen, we can actually
unlock not only access to video games, but all sorts
of visual experiences in daily life.
Speaker 2 (03:27):
I'm afraid from Kaleidoscope and iHeart podcasts.
Speaker 4 (03:34):
This is kill Switch. I'm Dexor Thomas.
Speaker 1 (03:40):
I'm sall right, good guy.
Speaker 2 (04:15):
How did you become interested in how blind people play
video games?
Speaker 1 (04:20):
It all started with a YouTube video that I actually
encountered while I was a PhD student. It was a
video of a blind person by the name of Terry Garrett.
He was playing a game.
Speaker 6 (04:33):
Called The Legend of Zelda Ocher Enough Time, which is
a classic Zelda adventure game that I remember playing as
a kid as well. Yeah, the interesting thing about Terry
Garrett was that he wasn't blind his entire life, like
he actually grew up as a little kid playing that
game and then later lost his vision, and as soon
as he lost his vision, he almost got shut out
of video games completely.
Speaker 1 (04:55):
For years, the only things he could play was some
games that he could find online would were called audio games,
which were games specifically designed for people who are blind
that are audio only.
Speaker 6 (05:06):
You know, in a lot of ways, they're very distilled
experiences compared to what sited players would be able to play.
Speaker 1 (05:12):
So one day Terry decided to pick up Zelda again
and say, though, I want to play this game.
Speaker 6 (05:17):
And I'm going to do it no matter what. And
it took a whole bunch of trial and error. He'd
do a whole bunch of like hacks that he figured out. So,
for example, he would walk around a room and whack
the walls with his sword again and again and again
until he heard a different kind of clinking noise which
told him like, oh, that's the door instead of the
rest of the wall, and so that's how I might
get into this room.
Speaker 1 (05:38):
There he'd memorize a whole bunch of details, and.
Speaker 6 (05:42):
That really landed for me because like it showed me,
here's something that i'd take for granted. These video game experiences, right,
and there are a lot of people who go through
so much effort just to have some semblance of that experience.
Speaker 1 (05:56):
Like I was you know, I was floored.
Speaker 6 (05:57):
I was amazed, But at the same time I felt pain.
I felt like, wow, like this is not a good
state of affairs right now. We've got to do better
than this.
Speaker 1 (06:09):
This isn't right.
Speaker 6 (06:10):
And that motivated me as like, Okay, here's a place
where just rethinking.
Speaker 1 (06:15):
How we're designing these games, these computer systems.
Speaker 6 (06:18):
Really can really enable people to have new types of
meaningful experiences.
Speaker 2 (06:23):
Yeah, you know what, We've gotten to the point where
we've legitimately got cars that drive themselves. And when there's
a kid who wants to play a videome, or an
adult for that matter, who wants to play a video
game who can't see the screen, we sort of shrug
our shoulders, I think, collectively, and say, you're asking a
little bit too much, buddy.
Speaker 1 (06:45):
Yeah, that's right.
Speaker 2 (06:46):
And you said no, you said no, I don't think
you are that's right. There are more games out now
that have accessibility features for blind and low vision players, like,
for example, there's SPORTSA and there's also the last of
Us too, which we mentioned in the last episode. But
that's a handful of games out of well a lot
of video games, and it's probably not fair to compare
(07:06):
the big blockbuster games to the smaller independent games, but
they do have something in common. Most games, especially the
blockbuster ones, the Triple A games what we call them,
are a lot further behind than they could be in
terms of blind accessibility and accessibility in general.
Speaker 6 (07:22):
Most mainstream games, the kinds of games you'd buy on
Steam or Xbox, PlayStation, Nintendo, are barely accessible to people
with disabilities. That includes people who are blind to low vision,
people who are deaf or hard of hearing, people with
motor impairments, people who have a hard time like mashing
buttons or pressing too many buttons too quickly.
Speaker 1 (07:44):
In sequence, that's the current state of things.
Speaker 2 (07:47):
Game companies are starting to take accessibility more seriously. Some
of this stuff seems kind of obvious, like the ability
to make text larger, or the ability to switch to
a color blind mode, or an option to read out
the text on the screen. But in order for a
blind or a low vision player to be able to
truly experience a video game, and anything close to the
way that say, I experience it, that game needs to
(08:10):
be built with that player's experience in mind from the start.
But unfortunately, the industry standard right now is to essentially
dumb down the game for that player.
Speaker 6 (08:20):
Oftentimes, the basic approach to accessibility is just to make
the game a little bit easier that actually take away
agency and choice from the player. So, for example, they're
video games that have accessibility features that allow players to
skip different sections or find items faster, but they do
(08:41):
so in a way that almost plays the game for
the player, like, oh, you can't really see the path ahead,
you can't really see the image.
Speaker 1 (08:49):
Well, we'll press this button and I'll just warp you there.
Speaker 6 (08:53):
And that form of accessibility does not offer players the
same type of experience, and that's the more challenging type
of accessibility problem.
Speaker 2 (09:02):
To solve this type of accessibility of making the game
easier to play and taking away some of the agency
of the player. You can even see that in the
Last of Us, and this is the game that's arguably
gotten the most positive attention for their accessibility features.
Speaker 7 (09:17):
What The Last of Us does is you can hold
a button to automatically move your character to the next
waypoint to the next object of interest.
Speaker 2 (09:26):
Vishnudnair is a research engineer and he was a PhD
student in Brian's lab, which started as just the two
of them working on video game accessibility technology.
Speaker 7 (09:35):
A lot of the players that we talked to called
it the golden path. That's all it does. We refer
to it as guidance. These are guidance based systems and
that's all they do. It's like you're taking their hand
and dragging them wherever, and that's not great for independence.
And it's that independence and autonomy, that ability to make
(09:57):
decisions on what I want to do, Which do I
want to turn that corner and maybe find a treasure
chest or if I turn that corner, maybe I'll encounter
a very powerful enemy and die. You know, making those
decisions and facing those consequences is like the most important
part of having fun, and being guided doesn't allow you
to have those experiences.
Speaker 2 (10:20):
This Golden Path will basically walk you from point to
point in the game, basically just getting into the next
spot to keep the story moving. It is well intentioned,
but it doesn't give blind players the same experience as
a sided player. This is gonna sound weird, but it
turns out that letting a blind player drive a race
car is relatively pretty easy. What's hard is letting that
(10:41):
blind player get out the race car and just walk around.
Speaker 4 (10:45):
But Brian and Vishnu think they got to figure it out.
Speaker 2 (10:48):
That's after the break. So let's go back to Terry Garrett.
That's the blind gamer who be Zelda sixty four. The
way he navigated the game was by going up to
a wall and hitting it with his sword, and based
on the sounds that he heard as he went down
(11:08):
the wall hitting it, he could tell if that was
just part of a normal wall, or maybe a door
or a secret passageway. This was a really clever workaround
for a game that was not meant to be blind accessible. Nowadays,
we have games that are developed with blind accessibility in mind.
They incorporate what Vishnu might call that golden path. So
what Terry was having to do was definitely tedious, and
(11:31):
a golden path approach is way more efficient. That sounds great,
but let's step back for a second. Does more efficiency
actually equal fun?
Speaker 6 (11:43):
You can imagine a game like Zelda right where we
have the main character Link and around Link might be
a ladder to a second floor, and then a door
into another room, and then a treasure chest or some
bushes that you can cut for rupees the game's currency,
et cetera.
Speaker 1 (12:00):
Objects around link at any given time, and.
Speaker 6 (12:03):
As a cited person, we can just move the camera around,
look around, see what those things are, figure out which
way we want to go, what we want to do next.
Speaker 1 (12:11):
But as a blind person, that experience is very different.
Speaker 6 (12:15):
Even with some of the cutting edge accessibility features that
are out there in games right now, the way that
that experience would play out is the blind player might
like press a button to just open a menu that
reads out the objects around them, so it might say door, ladder,
treasure chest, bush, whatever, like that's it. It's just a
(12:36):
linear menu, and oftentimes that menu kind of waters down
the experience of looking around and exploring. So, for example,
if the whole point is to find like a small
key hidden somewhere in some nook or cranny that might
unlock a door on another side of the dungeon, if
you pulled up the menu and the menu just said
small key, it kind of spoil the.
Speaker 2 (12:57):
Surprise, right, yeah, yeah, you not playing a game. It's
like you're on a guided tour.
Speaker 6 (13:03):
Exactly exactly, and so that experience of like exploration and
discovery is missing from those current accessibility tools.
Speaker 2 (13:14):
But how do you define that experience? So this is
gonna sound totally unhinged, but hear me out. There's this
YouTuber who has this series where he goes to what
he calls unremarkable and odd places in Zelda, like places
where you just go and there's a couple of rocks
near a river or something. Has nothing to do with
the progressing of the game. There's no mission critical item there.
(13:34):
It's just a river and some rocks. And one of
his videos has three hundred thousand views, and people in
the comments are saying, yeah, I remember that spot.
Speaker 4 (13:42):
I love that spot. I like hanging out at that spot.
Speaker 2 (13:45):
So how do we give blind people the closest thing
to getting a game and exploring, not accomplishing things, not
checking stuff off of a checklist, but playing the game
on your own terms. Brian in his last are taking
this question very seriously, and they published two papers on
applications that facilitate precisely this ability. One application is called
(14:09):
navstick and the other it's called Surveyor.
Speaker 6 (14:13):
So both Navstick and Surveyor are about making three D
adventure games more meaningfully accessible to blind players. So what
vish news work was really about was what's important to
the experience of exploration and discovery? When cited people are
playing Zelda? What do they do at one moment in
(14:34):
time the next moment of time, How are they having fun?
What choices are they making at any moment in time,
and like, what information are they exploring to figure that out?
And now, how can we make that process of exploration
and discovery more meaningfully accessible?
Speaker 2 (14:51):
The first step was figuring out how a blind player
could explore their immediate surroundings so that they could understand
what choices they had without just doing something for them
or telling them where the thing that they were looking
for was.
Speaker 7 (15:03):
We were like, okay, a person kind of rotates their
head all around them to look around themselves and get
a map of what's imediately around them. How do we
give that ability to someone with blindness? And so that
was where navstick came along, where it was really simple.
We just repurposed the right thumbstick on the standard game controller,
(15:24):
and all you have to do is just point the
right thumbstick in some direction and whatever lies in that direction.
The game's going to tell you that was it.
Speaker 2 (15:33):
So I tried navstick and it's really intuitive. Basically, you
move your right thumbstick around and whatever you pointed at,
a voice plays and tells you.
Speaker 4 (15:42):
What it is. Clean, Red Dirk Chocolate, Chocolate, Chippickies Better.
Speaker 2 (15:47):
This is a clip of Vishnu demonstrating navstick in a
virtual grocery store setting. You can hear the sound of
the voice moving around depending on where the item is
in relationship to you.
Speaker 7 (15:57):
Christians Coffee Grounds, Dirk Chuck. We actually created an entire
video game to test this, and everyone loved it. What
was amazing was that, you know, we had we put
in all these levels. All these levels had like different
trees and enemies that they had to shoot using a
laser gun. Every participant completed every level of that game,
(16:18):
no matter how complex it got. And what we found
was that everyone crapped at different strategies to get through
those levels. Maybe they went this way to the left
and defeated a few enemies to get to the finish line,
or maybe they just barrel straight through. They were able
to make those decisions and everyone said that, oh, this
is like just so much more fun than using like
(16:38):
a menu for instance, and just being told where to go,
I was able to like actually feel like I was
playing the game.
Speaker 2 (16:46):
This is a pretty simple application, but it's very powerful.
So I've tried doing the thing where you close your
eyes and make a mental map of your surroundings and listen, man,
I suck at it. This is something that blind people
are usually very good at because because they have to be,
but it can even for them get kind of taxing.
Navstick means that you don't have to memorize a brand
(17:07):
new area when you first go to it. You can
check where you're at right then. But for a full
three D game, especially a big game, napstick is just
one part of that equation.
Speaker 7 (17:18):
The question beyond navstick is how do we allow a
blind user to experience a larger area. You have this
entire dungeon that you have to explore. How do you
handle that? How do you break away from that guidance
paradigm where it's like you're leading a person by the
hand and leading them to the treasure chest. In game
(17:41):
design theory, discovery is one of the main elements of
video games that are important for a good experience, and
so how do we facilitate that, and so we create
this entire system Surveyor where they would use navstick, but
in addition to navstick, we help keep a law of
where they've been. Basically, if you were to take an
(18:04):
overhead view of a dungeon and you color in the
places where the person has been, the point is you
want to bring a user to the edge of the
area those borders between where they have explored and where
they haven't explored. That's what we did with Surveyor, where
effectively it would say, okay, about fifty feet away, there
(18:28):
is an unexplored area, and if you select this, you'll
be guided there. And once you reach that unknown the
guidance cuts off and you're left using knapstick to look
around that area.
Speaker 2 (18:42):
Oh so, it's kind of like fast traveling a game. Exactly,
I've already explored this part. Don't make me run across
a field for twenty minutes. I've already been here, I've
completed this area of the game. Let me get back
to the town. Cool, and it takes you to the
edge of where you've been, drops you off there and says, okay, figure.
Speaker 7 (19:00):
It out exactly exactly, and that's what promotes that sense
of autonomy. That's what promotes that sense of discovery. What
we're trying to do is, Okay, we don't need to
confine the player anymore, right if we can just open
it up, just take something like Tiers of the Kingdom
and put a player with surveyor in it. Can they
(19:21):
just explore that area, these cities, these little dungeons and
all that.
Speaker 2 (19:26):
Zelda Breath of the Wild and its sequel, Tiers of
the Kingdom are two of the biggest open world games
that are out right now. They're really popular, and it's
not just because they're good, but because they're really approachable.
I have friends who don't play a whole lot of
video games, but you know, they'll turn on the switch,
They'll jump in Zelda, They'll go run around in a forest,
not even fighting enemies, just having fun. Zelda Tears of
(19:49):
the Kingdom. You think you could put this onto a
Zelda Breath of the Wild type game and people be
able to play it.
Speaker 7 (19:57):
Perhaps, I mean, they are obviously going to be some
design decisions that to be made, But in my defense presentation,
I literally use Tears of the Kingdom as an example.
Speaker 2 (20:06):
Again, blind people can play some video games now and
feel that sense of satisfaction of accomplishing a task, like
finally beating that last boss. But that's not all a
video game is. I mean the freedom to just run around, explore,
hang out, that isn't there for a blind player. But
now with these tools that Vishio and Brian have developed,
a blind players, say like Ross Minor from last episode,
(20:28):
could probably do precisely that, just go wander around without
assistance or special modifications. That potentially opens things up a lot.
Speaker 7 (20:38):
Yeah, that opens a lot of social opportunities as well.
If we can open up the same experiences to blind
players as side of players have, then we can foster
better connections at the end of the day.
Speaker 2 (20:53):
I mean that totally makes sense. I mean, you know,
if I found out somebody played Earthbound, we're immediately friends immediately,
you know what I mean.
Speaker 4 (21:00):
Or same thing with really.
Speaker 2 (21:03):
Anything, music, movies, books, whatever, You've experienced, the same thing,
you read, the same stuff, you listen to the same music.
Obviously video games figure into that pretty heavily too, right,
So you may think this all sounds really exciting, but
you're wondering when this stuff's gonna come out, or you
might think that this is an awful lot of effort,
(21:23):
and you're wondering why these brilliant researchers are wasting their
time on video games. Well, I got two answers for you.
That's after the break. The tools that Brian and Vishnu
(21:45):
were talking about are pretty incredible, but they're also really
intuitive and simple. It kind of makes you wonder why
stuff like this is so rare. Well, A big part
of the problem is that game companies are really siloed.
They don't talk to each other, and they definitely aren't
willing to share their secrets with each other. I've had
numerous people tell me who develop games, either at a
(22:07):
big company or even especially if they're an independent developer,
They'll say, look, I would love to make my game accessible.
It would take me six months. I'd have to stop
everything I'm doing and I'd have to figure it out somehow.
And realistically speaking, that six months and I actually can't
afford to do that.
Speaker 6 (22:25):
Yeah, And I think that the reason behind that is,
like every studio has to build these things from scratch.
Speaker 1 (22:33):
There aren't enough of.
Speaker 6 (22:35):
These like accessibility widgets that are available like just kind
of as a package, ready to go that you can
plug in.
Speaker 2 (22:42):
So that's exactly what Brian and Vishnu had been working
on within the lab.
Speaker 6 (22:46):
We're actually creating a plug in for Unreal Engine to
make it easier for any game built on Unreal Engine
to adopt napstick, and then we'll later add survey or
functionality to that.
Speaker 2 (23:00):
Reel is a game engine that a lot of Triple
A games, that's those big blockbuster games are built on.
Speaker 4 (23:05):
So an engine is.
Speaker 2 (23:07):
Kind of like the base program that a lot of
game developers build their game on top of. So, for example,
if you're making an adventure game where you walk around,
if you had to program in the concept of gravity
or the concept of a ground that you walk on,
you'd never get anything done. So you get an engine
like Unreal that has a lot of the basic stuff
built in, and if you need additional features, you get
(23:29):
a plug in. So there are plugins that'll help you
make realistic trees or anime an ocean or a river,
or make realistic sound effects. And what Brian is hoping
to do is to make a plug in that any
game developer who's using Unreal could just bring into their
engine and add the navstick or the surveyor features.
Speaker 1 (23:48):
So we are going to put as a law. We're
going to publish it on Unreal Engine marketplace so.
Speaker 6 (23:52):
That developers can see the possibilities and see exactly like
how to implement and incorporate these things into the game.
Speaker 4 (24:00):
How much would this cost for a developer.
Speaker 1 (24:02):
It would be minimal. I don't expect it to cost
more than say, like one hundred bucks or something like that.
Speaker 4 (24:08):
Okay.
Speaker 2 (24:09):
Yeah, And the ability to throw that in and potentially
have some more people by your game who would never
have picked it up is kind of cool.
Speaker 6 (24:17):
Absolutely, Yeah. And I would love to, you know, for like,
especially like independent developers and things like that. You know,
I'd love to have a program where they can they
get incorporated into their game without having to pay. This
is something I'm still working with the university on.
Speaker 2 (24:34):
The hope is that these plugins make it so obviously
easy to add in accessibility features that developers see it
and they say, you know what, let me just download
this plug in and just do it, and they'll be
able to get started pretty soon.
Speaker 6 (24:46):
We're actually hoping to launch that by the end of
this calendar year twenty twenty five.
Speaker 4 (24:51):
Wow.
Speaker 2 (24:52):
I really do hope that game developers check this out
because I'd love to see what kind of games come
out in the next few years that use this technology,
because this isn't just a niche feature. It really can't
open up games for an entire community of people who've
been shut out of video games, and that can open
up new ways for everyone to connect. I remember early Quarantine,
(25:13):
my niece was in a different state, and when we
wanted to communicate, she didn't call me. She said, Yo,
let's get on Fortnite, and that's how we talked. We
got in Fortnite and we ran around and build stuff,
and you know, she showed me some weird little structure
that she'd built and her friends are running around in there,
(25:34):
and like, that's how we talked. And so video games
now are, for better or worse, where people spend time
with each other.
Speaker 1 (25:44):
If they're immersive, right, it's a form of immersion.
Speaker 6 (25:47):
And so I think that, like you know, any tool
that can can enable access to video games could also
enable access to a whole host of immersive experiences as well,
like being able to like tour apartments, for example, without
having to visit that apartment or two or a different city.
Speaker 7 (26:04):
There's potential to bring nastick something like nastick into the
real world as well. Really yeah, navigation through the real
world using you know, you have a blind person outside
or in a building, you know, pointing in some direction,
maybe with your camera or something else, and having it
say what what's around you.
Speaker 2 (26:26):
This is a pretty interesting idea because it could be
an augmented version of what blind and low vision users
have found in the meta smart glasses. If you haven't
seen our episode on that, you can check that out.
But instead of AI as most people think of it,
this would be layering a video game kind of on
top of reality.
Speaker 1 (26:47):
An example that I like to give is New York's
Pen Station.
Speaker 6 (26:51):
It's the busiest train station in the world, and it's
massively complex to someone who's never been there before, with
multiple railroads and subway lines and you know, et cetera.
Speaker 1 (27:01):
It's huge.
Speaker 6 (27:03):
And imagine being able to like swipe by like tapping
the side of an earbut and exploring kind of circularly
and being able to hear spatial audio of like what's
around you at Penn Station.
Speaker 1 (27:16):
Oh, the ticket office is here, that's track twelve, that's
track thirteen.
Speaker 6 (27:20):
Here's the McDonald's, or here's the shake shack, right we
have this physical world, we just have to create a
virtual representation of it first.
Speaker 1 (27:28):
Let's say like a video game version, a digital version.
Speaker 6 (27:31):
Oh so that like now, when we're in the physical world,
if the system knows where you are, it's the same
as napstick. You're exploring that environment, but you just happen
to be physically inside the place as well.
Speaker 2 (27:43):
The video game stuff that I'm messing with and that
in the future gamers will be able to use ends
up kind of being like a trial run for using
this in real life at a train station or a
grocery store.
Speaker 6 (27:56):
Yeah, that's a great way of thinking about it. Oftentimes
we think of the games that we prototype as being
almost like testing grounds, Right, that's our little our peatrie
dish really to test our systems. But it's just a
fake version of what might be a real world environment
as well.
Speaker 2 (28:18):
If you told someone in twenty fifteen that they wouldn't
be able to walk to their mailbox without being recorded,
they would think that you were describing as science fiction dystopia.
If you say that now in twenty twenty five, they'd say, well, yeah,
you're just describing the basic function of my neighbor's ring
camera we now expect technology to keep expanding, so what
if we started expecting accessibility to advance also. I mean
(28:41):
five years ago, I thought Brian was nuts for thinking
that Ross Minor would be able to play a racing game.
Speaker 4 (28:47):
And then this year I.
Speaker 2 (28:48):
Walk into Ross's apartment and He's got a driving wheel
on his desk.
Speaker 4 (28:51):
To play Fortza.
Speaker 2 (28:52):
But this stuff doesn't just happen. It's the industry, it's
the universities, and it's the gamers. I'm not saying you
need to write a strongly worded email to Nintendo, but
you do have influence over things, and they do affect everyone.
I mean, look, I personally do not need Knavstick and
I don't need Survey Air, but I am more likely
(29:12):
to buy a game if I know that it means
that I might get to play co op with Ross.
The stuff that people like Brian and Vishnu and Ross
build is gonna make games more interesting for everyone. I mean,
who knows, Maybe someone listening to this right now, maybe
you could end up contributing something to that process. And
(29:34):
thank you for listening to kill Switch. This was a
fun one to do, and hey, if you got any
ideas for the next one, you can hit us on
email at kill Switch at Kaleidoscope dot NYC, or we're
on Instagram at kill Switch Pod. And if you listen
all the way through our first double episode, hopefully that
means you liked it. So you know, get that phone
out and please leave us a review. I know people
(29:56):
say it all the time, but it really helps other
people find the show, which he'll us keep doing our thing.
Killswitch is hosted by Me Dexter Thomas. It's produced by
Sena Ozaki, Darluck Potts, and Julia Nutter. A theme song
is by me and Kyle Murdoch and Kyle also mixed
the show from Kaleidoscope. Our executive producers are Ozma Lashin,
Mangesh Hatigador, and Kate Osmore from iHeart Our executive producers
(30:21):
are Katrina Norvil and Nikki Etour. Oh one more thing.
Oh this is for all my game developers out there
who are listening to this. You might have been a
little disappointed to hear that they're making these plugins for
Unreal Engine, because look, Unreal is great. A lot of
blockbuster games are built on it, but it's just that
it's kind of for the big time games. You know,
why would you leave the independent game developers out, the
ones who you know, use different software, cheaper development software
(30:43):
like say Unity. Well, I asked Brian about that.
Speaker 6 (30:47):
I wanted to follow the model that happened with Fortsa,
where like a major developer had this feature incorporated into
their game that got noticed by the whole industry. That's
an important and first domino to fall. And so we
do want to make a Unity equivalent of the plug
in as well. But we chose Unreal first for that reason,
(31:08):
so that we could work more closely with some of
the bigger developers, work through all the kinks with their
larger teams they would have more resources, and then figure
out how to tailor both that Unreal Engine plug in
and how to do things right on Unity for the
smaller indie developers the first time.
Speaker 2 (31:28):
So there's your answer, and you know I get the
approach anyway, Catch on the next one, good guy,
kill switch News
Hosts And Creators
Oz Woloshyn
Karah Preiss
Show Links
AboutPopular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy, Jess Hilarious, And Charlamagne Tha God!