August 10, 2016 • 42 mins
How could haptic feedback motors improve the boots astronauts wear? And what are some other applications?
Learn more about your ad-choices at https://www.iheartpodcastnetwork.com
See omnystudio.com/listener for privacy information.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
Brought to you by Toyota. Let's go places. Welcome to
Forward Thinking. Hey there, and welcome to Forward Thinking, the
podcast that looks at the future and says these boots
are made for walking. I'm Jonathan Strickland, and I'm Joe McCormick,
(00:21):
and our other host, Lauren voc obamb is still unto
the weather, but she will be back with us next time.
I'm quite sure. Yeah, pretty pretty pretty positive on that.
And don't don't worry about her. She's doing fine. She
just got this scratching through. It's a bit of a
shame because she's the one who actually pitched this episode topic,
wasn't she. I think so so, Jonathan, I have a
question for you. Asked me, did you remever did you
(00:42):
ever own a pair of moon boots? I I can
honestly say, with on conviction that I have never owned
a pair of moon boots. You're aware of what I'm
talking about. That not not the actual boots that astronauts wear.
I'm so you mean the style of boots known as
moon boots. Yeah. No, I I had never owned a
pair of those. I have warned things that I suspect
(01:05):
we're equivalent to moon boots, at least in their the
clunkiness factor. But I personally have never owned any Oh
that may be what I had when I was a kid.
I had some puffy boots that I referred to as
moon boots. I don't know if moon boots is a brand,
that a brand. Well anyway, I don't know one way
or another. But I had some puffy boots that were
(01:26):
quite beloved to me, and I wore them out in
the snow, and I wore them out in the summer
also just because I thought they looked cool. Wow. Okay, well,
you know it just tells me a lot about you, Joe,
and and be proud of who you are. But we
wanted to talk today about a specific type of of
approach to designing shoes for astronauts specifically, but there are
(01:51):
implications beyond that that addressed some problems that they've encountered
in the past. So let's let's talk about why we
would even have this conversation in the first place. What
is the big issue? What is the problem astronauts face.
The problem is astronauts are always too intoxicated to walk
(02:13):
around properly. No that's not true, No, not at all.
That we we are. But however, if you were to
watch certain videos, you might get that impression. Because Jonathan,
did you watch the astronauts falling over? So did I? Even?
I even have a little little note that I put
on there. Okay, So Joe decided to kind of watch
this video and then he shared the video in our
(02:36):
notes of astronauts having difficulty standing upright while walking about
on the Moon's surface. And that's really what this episode
is all about, Like, how can people design uh a
system that allows astronauts to walk around on a surface
(02:56):
like the Moon's surface without the danger constantly falling over,
because based upon the YouTube video you've sent, it looks
like that is a common issue. Well, it's kind of funny,
isn't it, Like they walking can be hard when you're
on the moon, you know, yeah, I I wrote. My
comment on the video was this is majestic. You really
(03:19):
get the sense of achievement and grandeur as you watch
chubby figures fall about in the dust. It really is
like watching little toddlers fall over in a sandbox a lot.
But it's not the fault of the astronauts. It's just
really hard to get around in bulky spacesuits. And once
you're down, it looks like it's pretty challenging to get
back up again. Yeah, they kind of try all these
(03:40):
little moves to sort of pop back up, but they
keep falling over and they look like crazy breakdancing moves
at some points. Because there's one point where you see
a guy who is holding himself up in a horizontal position,
like like parallel to the Moon's surface, on his hands
and just trying to push himself up so they can
get his feet up underneath himself. It's pretty incredible. And
(04:03):
of course this isn't just the Moon. This is presumably
true for other space environments like Mars and other foreign
planets as well, where asteroids as well. That kind of
probably even harder on asteroids, the lack of i mean
almost you know, no negligible gravity. Yeah yeah, so, uh
so space suits are bulky their pressurized environments, and they
can tend to cause a feeling of perceptual segregation from
(04:27):
the outer atmosphere. You know, you're like you're living in
this interior world and just sort of vaguely bubble buoying through. Yeah,
like your perception is thrown off. Yeah, so they have
to be bulky, of course, because if you know, they
have all these built in protections, shielding regulatory systems. But
this often leaves the astronaut with a sort of limited
(04:48):
ability to sense their surroundings both by sight and by touch.
Right and then with that reduced gravity that I would
imagine adds another element of uncertainty and and and kind
of clumsiness to the whole thing. I imagine that walking
around on the Moon is a lot like have you
ever gone up a set of stairs that you've gone
up and down a thousand times and you're not really
(05:11):
paying attention, and in your mind you're thinking there's either
one more or one fewer step than actually are there
on the stairs, And that is a a really alarming
thing to have happened to you that moment. You either
take an extra step on a stair that's not there,
or you fail to take that extra step and you
(05:33):
miss a step that actually is there. I argue often
that it's worse going down than going up, Like I've
tried to step down at the bottom of a set
of stairs when I thought there was one more to go,
and it is the most jarring experience to try and
step down where you cannot because there is solid floor there, though,
I imagine it is much more jarring to step down
thinking you should be experiencing about you know, point eight
(05:56):
more g s than you are, right, and then you
end up falling over as a result because you have
either underestimated or overestimated. You couldn't really tell when you
were making contact. You didn't really know when you could
shift your weight, and so you either do so prematurely
or you're not quite ready to do so when you
need to. All of these factors come into play, and
(06:17):
it explains why we get this wonderful sequence of astronauts
falling over as if they're just completely helpless on the
surface of the moon. But of course, when you were
on a literal moonwalk falling over, despite how funny the
video is, that's not a good idea. You don't want
to be doing that. Number One, space suits are pretty
well shielded, but there's always a chance that you could
(06:38):
damage or puncture something vital. That's just pretty much everything
on the space suit. Yeah. Then on top of that,
there's just the waste of limited time and oxygen resources
lying stuck on the ground like this marshmallow man turtle thing. Right,
If you have a specific set of goals for your moonwalk.
You don't want to get stuck for ten minutes trying
to get up. Yeah, becausebviously, whenever any of that stuff
(07:01):
gets planned out back on Earth in the initial stages
where you've you've got your mission parameter set, you're gonna
build in extra time. Right, You're never going to say,
let's let's really cut this to the wire, right that
that makes no sense. However, if you fall over, then
you start eating up that extra time that was built in,
and and this is a real concern. Yeah, So the
(07:22):
story that made us want to do this episode today
is about a particular response to this kind of problem.
So the story refers to some work done by M I. T. S.
Department of Aeronautics and Astronautics or Aero Astro and the
Charles Stark Draper Lab in Cambridge, Massachusetts. I love the
fact that's in Stark Labs. Yes it is, but they're
(07:46):
not that Stark because he works alone. It's a Stark contrast,
I imagine it is. Now. So they've been working on
this potential solution and it is space boots with environmental
swerves and embedded haptic motors. So it works like this.
Imagine you are an astronaut on the surface of the
(08:07):
Moon and you're shoveling up some barrels of lunar regular
to transport back to Earth, where it has become a
wildly popular flavoring agent in hot cuisine, of course. But
of course you are walking through a shallow dip on
the lunar surface and you feel a sudden vibration on
the outside toe of your left foot, so you stop,
you take a step back, You look down, you see, oh,
(08:28):
there it is. It's a rock, or maybe it's a
little room bus sized robotic rover sitting deactivated in your way,
whatever it is. Having avoided tripping over it, you go
around scoop up some more of the ultra fine particles
of cash money. Pretty simple, right, So you don't have
to depend on your eyes. You've got a tactle sensation,
(08:48):
this vibration that warns you to stop moving and reach
out your course. And that's exactly what we mean when
we talk about haptic feedback. If you're not familiar with
the term haptics, it really means engaging your sense of
touch in a way that gives you more information about something,
whether it's your environment, or a virtual environment or something
(09:10):
along those lines. A very basic example of this is
you likely have a smartphone or cell phone that has
a little motor that vibrates when you get a call
or a text. That's a haptic feedback system telling you, hey,
you've got an alert. So maybe it doesn't make an
audible alert so that you know if you're at work,
(09:31):
you're not having your phone go grouper or whatever. Well,
here's one. If you have a touch screen keypad on
your phone, very likely the phone vibrates ever so gently
when you correctly press a key, so that so that
you know you did hit the key instead of sort
of going off the side and not registering the the
input right, or that you just start typing you know,
(09:53):
h h h h h h h for hello, because
you haven't figured out if you're actually, you know, making
that contact with the screen or not right. And I
found out when when my phone's power saving features go
on when it's low on battery and it stops doing
that that response. Yeah, I find it disturbing when I'm
typing with my fingers, I'm like, wait a minute, am
(10:15):
I am I typing? I don't know if the letters
are going Yeah, I've gotten used to it. I had
turned mine off when I was at home, um after
a certain time, Like I could, I could create parameters
of when haptic feedback was turned on or turned off,
and that was specifically so that I didn't disturb either
my wife or my dog whenever I decided to look
(10:36):
on something on my phone and look something gup, because
every every time I would type, you get this little
haptic buzz and it would be distracting. But I also
found it very difficult to navigate on my phone without
it because without that feedback to reassure me that yes,
indeed your touch has been has been registered, I felt
(10:57):
like I wasn't properly activating my phone. So haptic is
a very powerful kind of disciplines of a very powerful
technology or strategy. I guess you could say to engage
our senses in another way and give us additional information. Yeah,
it's a it's a way of repurposing information that you
would get through one sense and feeding it through a
(11:19):
different one. So in July of this year, researchers presented
their findings on on these haptic feedback boots boots for
space at the International Conference on Human Computer interaction, and
specifically they've been studying what types of sensations applied to
which regions of the foot do the best job of
(11:41):
keeping people from tripping over things, or actually that that
part would be inferred, because really what they're looking for
first is like when people can feel the sensation and
how well they can feel it right, and and can
they identify from what direction it's it is coming? So
that way, like if we were to have the outer
left edge of your foot, uh, buzz, would that translate
(12:05):
in your mind, Oh, there's something forward and to the
left of me that I need to look out for. Yeah, uh,
And and there are lots of different options for how
you could provide haptic feature right. So you know, our
nervous systems are kind of quirky for reasons of evolutionary happenstance,
and so they respond better to some stimuli than to others.
So they actually looked into what works best. So scientists
(12:29):
Alison Gibson, Andrea Webb and Leah Sterling conducted this study
to find out exactly how good people are at detecting
different kinds of sensations on their feet under various conditions.
On my feet, my feet were tingling reading about this. Yeah,
and it's interesting to hear about the experiments because they
included everything from kind of a straightforward approach where they said,
(12:50):
are you gonna feel a buzz? I need you to
tell me, like, on a scale of maybe one to ten,
how strong was that buzz and where on your foot
did you feel it? And then they start and making
it a little more complicated by making people like do
things while they're administering the buzzes and try and see like, well,
can they still accurately describe the intensity and the location
(13:12):
because this obviously that would be important. You know, if
you're walking around the surface of the moon, you're probably
not just completely focused on how are my feet feeling
right now? No, you're trying to get that delicious regulars
back to your back to your transfer. Emerald will not
wait for his for his moon spice bam. So if
you want to look up their their presentation, it's it's
(13:33):
called user abilities in detecting vibro tactle signals on the
feet under varying attention loads. Vibro Tactle is amazing, what
a great word, And yeah, varying attention loads obviously that's
the reference to what I was talking about earlier. But
you know, sometimes you have to create some distractions, nothing
of real like importance or or urgency, but rather just
(13:57):
something to occupy the person's mind. So that example is
one of the independent variables they tested attention state whether
you're distracted by other stuff. Another one was vibration locations
of tow or heel in step or outside, etcetera. And
then the third was vibration signal type. And this is
not correct, but I wish this meant good vibrations versus
(14:19):
bad vibrations bad vibes manuh no. In actuality, the different
vibration types were high, low, increasing and decreasing, increasing and decreasing.
Matter because of something I'll get to in a bit, well,
well before before we get into that, just for a
clarification on my own part, because this was something that
I missed when I was reading it. When they say
(14:39):
high and low, are they talking about frequency or intensity? Oh? Well,
actually I think it's intensity, But I'm not positive because
I was it's intensity. I would think so too, because
to me that would just mean like, is the motor
vibrating at a greater intense stealer as in, is it
making heart of great bigger vibrations is the ample alitude
(15:00):
increased as opposed to is the frequency increase. I would
imagine the same thing. That was my interpretation, but I
didn't actually read Yeah, okay, co uh So these could
be delivered to six different places on each foot and
that results in twenty four unique different kinds of signals.
And then they also, like you said, they ran a
(15:20):
distraction test, so they were like, let me know if
you can feel anything on your feet and uh and
what it is, where it's coming in, and what it
feels like. But then also do that while I show
you a random number and you start to count up
by threes Jonathan twenty four, thirty thirty three, thirty six.
I gave you an easy one. Let me get a
hard no no, no, no no, don't don't have a
(15:43):
liberal arts major. Uh So, But it turns out even
with a pretty simple test like this subject has had
some difficulty. They they had trouble identifying when the vibration,
for example, was steadily increasing, so that that's not just
the on and off high and low sensation, but when
it's slowly getting more intense and even in the undistracted condition,
(16:06):
so even when they're not trying to do the counting.
People did not do a great job of detecting when
vibration was decreasing in intensity. Uh. And also no surprise here,
but some parts of the foot are just more sensitive
than others. You know, the middle of the outside of
the foot was pretty dumb. By the way, A little side. Now,
do you ever play that game where you touch somebody's
(16:27):
back with different numbers of fingers and asked them how
many it is you should you should agree beforehand with
the person you're playing, right, there's consent is absolutely don't
don't surprise somebody with this game. I've I've played variations
of that, like as a kid. There's also the I'm
gonna draw something on your back, tell me what it
was you I drew and I, uh, it's difficult. Yeah,
(16:50):
people don't realize how dumb their backs are. The tactle
neurons back there is just I don't know, you know,
I'm touching your touch somebody with three fingers and they're like,
oh one right, yeah, let's say. And so you think
about it, like, there are definitely gonna be areas of
the foot that will be more responsive to to haptic
feedback than others. And when they were talking about increasing
(17:13):
or decreasing the intensity, I could easily imagine that being
a difficult thing to uh, to detect even so much.
Is like, even if I have a phone in my
hand and I'm increasing the motor intensity until you get
to like unless it unless it does does it in
big jumps, the differences can be fairly subtle, And then
(17:35):
I think, well, what if I felt that on my foot,
and what if that were the important determining factor. Like,
as I approach a potential stumbling block, the intensity increases. Uh,
if it is to settle, then that becomes kind of useless.
It just is going to be a constant distraction as
I look around, and maybe I can't identify what the
(17:56):
thing is because it's not quite close enough for me
to notice that it is something I'm going to stumble over. Uh.
And then next thing, you know, I start to ignore
this this buzzing because so frequently it's not helpful, and
then I'm falling all over again. That exactly is a
problem people are worried about. Okay, so how do you
keep people from tuning out the feedback? Yeah, you have
(18:17):
to have it be urgent enough that it's meaningful, but
not so urgent that it is that it's misleading right there.
It's a delicate balance, but you can see where it
could become really useful if you have the sensors in
a boot and you have the alert happen where it's
(18:37):
not gonna be so sensitive that it's it's it's set
off by everything, because then you're just like, well, all
I can tell you is being on the moon is
crappy because all my feet are just buzzing. These dogs
are buzzing. In other words, that's an alternate title for
this episode. Yes, so of course this technology it's still
(18:58):
got a lot of ground to cover as the study,
but a lot of moon ground. Because while a simple
on off vibration might be somewhat helpful, one thing about
the world around us is it's best represented by continuous data,
not on off analogue rather than digital. Yeah, so the
the increasing and decreasing quantities would probably be the best
(19:19):
way to represent something like proximity or even like the
the size of the potential obstacle, because again we have
you have such a limited field of view inside a
space helmet, right like it cuts off your field of
view significantly. Uh, you might want to use intensity to
indicate perhaps not the distance to the obstacle, but perhaps
(19:42):
the the size of the obstacle, like whether or not
it's a sizeable rock that's ahead of you that you
might stumble over, or a drop down that's been you know,
that's your boot has uh detected where you know you
don't want to have that next step forward be four
inches lower than what you had anticipated. Yeah, exactly. So
(20:03):
the mn T news piece that covered this particular research also,
I thought this was interesting. Spoke to a doctor, Shirley
right Dick, professor of Health and Kinesiology at Perdue, who
specializes She specializes in like falling down. Basically, so do I,
but not not. I don't have the credentials. I just
have the experience America's Funniest Home Videos professor. And but
(20:28):
she pointed out that this would be useful for more
than just astronauts. I think that that's clear. But for who,
Well she gave the example of how about firefighters. So
for firefighters, you go into a burning building, there's flames,
there's smoke. Environmental navigation inside a burning building is a
well known problem. Uh, And so this could be very
useful for them too, And I imagine maybe deep sea divers.
(20:51):
Maybe you know there there are lots of conditions where
you don't want to be tripping over stuff and your
personal visibility is going to be limited. And the even
beyond that, we started talking and in fact articles have
also been pointing this out. It's not just Joe and
I have come up with this idea, but we've talked
about how these sort of haptic feedback systems could be
(21:11):
used for people in everyday situations, particularly people who may
have uh problems with their vision. Yeah, exactly, So one
obvious application would be for people with visual impairments. So
at a certain level, this is harder than it sounds,
because you can give somebody a vibration in their shoe
(21:31):
if they're about to kick something, But how do you
communicate more complex navigational information about a person surroundings without
visual information. The thing about the astronauts is they should
be able to see generally what's going on around them.
The problem is they might not notice there's a rock
at their foot. But but imagine you're completely blind and
(21:52):
you're trying to use haptic feedback to give people navigational
information about what's around them. You can you can give
a kind of dumb and form, but it gets more
difficult the more info you're trying to encode, right, So,
so indicating that there's a possible surface that you're going
to come into contact with right there, that's that's one thing.
(22:14):
But to get to a level like the way, how
about there's a street and there are cars on it
right right. One of the things I thought about when
we started talking about this is it's a a step
toward spiky sense, but perhaps a better representation would even
be Daredevil's heightened senses. That Daredevil's capable even though he's blind.
(22:34):
Matt Murdoch spoiler alert is blind, he's capable of sensing
through this this crazy heightened ability to almost see vibrations
like he can he can sense sound at such a
level that he can and he knows shapes of things
and whether something is coming towards him or moving away
from him. That level is well beyond a little motor
(22:59):
vibrating against you to indicate that there's something you might
be bumping into in the very near future. And um
and while I think that this technology definitely has the
potential to increase a person's environmental awareness, particularly someone who
perhaps has has just become blind, who hasn't had hasn't
(23:21):
been dealing with an impairment over the course of years. Uh,
this might be something that is more helpful. Yeah, and
this is something that is also under development. In fact,
more m I T research. There. There's research on the
hardware behind a wearable device that uses three D camera
data to provide navigational feedback to people with visual impairments.
(23:43):
It's been known as the Virtual Guide Dog. Interesting. But
you know, Jonathan and I were talking before we came
in here about the idea of of expanding this even
farther with something that Jonathan dubbed pedestrian assist. Yeah, kind
modeled after the driver assist systems that were familiar with
(24:03):
things like you know, the lane assist, parking assist, breaking assist, acceleration,
you know, cruise control. But in this case, obviously I'm
not talking about your your shoes magically taking over for you, Like, hey, shoes,
I want to go to the mall and they just
start stepping and you're just like, hey, I can go
to sleep now. Yeah, So it would be self powered,
but it would be uh, sort of like a not
(24:24):
self driving car for your feet, but self navigating car
for your feet or or at least giving you Yeah,
that driver assists that warning accident prevention right like you're
going to hey, buddy, stop playing Pokemon Go for three
freaking seconds. That is a thing, because you're gonna run
into that street lamp. People have walked off of ledges,
haven't they? I I have personally like I I play
(24:46):
Pokemon Go, so full full disclosure. I play the game.
I do not disclosing your conflict of interest, Yes, exactly, well,
I want to make sure that people know that I'm
not making fun of folks who play Pokemon Go. I'm
making fun of myself, a person who does play Pokemon Go.
So subtle difference. I have made it a point when
I started playing, and i've I've stayed true to this
(25:09):
that I will not pay any attention to my phone
whenever I am crossing streets, because I know I would
be that person who stumbles over a curb and causes
danger to myself, to others, to you know, giving drivers
a heart attack, maybe possibly suffering a massive injury or
perhaps even death because I'm not paying attention to what
(25:31):
I'm doing, Because I am aware of this, and I've
had situations where I have made contact with stationary objects
because I was looking down at my screen and not up.
Now in in my case, that stationary object was a
tree branch that in my neighborhood. There's this one tree
where the branches is slightly uh lower down to the
(25:53):
ground than my height is. So if I don't duck,
I will bang my head against it. But if I
if I'm looking down towards my screen, it's out of
my view, my field of view. So I have hit
on more than one occasion the top of my head
against this branch. Now, it hasn't always been because of Pokemon.
Sometimes it's a real monster, that being my dog. So
(26:16):
I'm often looking down at him, and I frequently nearly
not quite close line myself. It's not quite low enough,
but I do get knocked right at the crown of
my head. Oh, I thought you were going to say
that you were on the way home and face timing
with him before. If my dog learns how to face time,
it will be the end of times. But because I
will never do anything ever again other than FaceTime with
(26:38):
my dog. But no, it is definitely one of those things.
Where if I were to have some sort of of
sensor on me and clothing, they could say hey, dude,
heads up, then I would not have that experience. Now,
in my case, that was a very minor inconvenience. I
was never hitting it so hard that I was causing
myself real injury. It was more more than anything else.
(27:02):
I was feeling embarrassed about clocking my head because I
wasn't really paying attention to where I was going. But
this could really end up being something that prevents injury
and death if if actually rolled out to and incorporated
in a larger number of of wearables. Yeah. And of course, Jonathan,
you were mentioning something about the way haptics would be
(27:24):
incorporated in uh in virtual reality and augmented reality environment. Yeah,
so this is not unusual either. It's something that a
lot of In fact, I would argue that the video
game world has pushed the haptic discipline forward more than
anything else as an effort to create more immersive experiences
for gamers. So a very simple version of that is
(27:46):
the rumble pack that you find in a lot of
game controllers, right, that gives you that So like like
when there's thunder in the game, the controller vibrates and
gives you that sensation something that you don't even really
necessarily register after you've played a lot of games with
that kind of functionality. But when it's gone, you're like, oh, yeah,
that is that does make a difference. Well, in virtual reality,
(28:07):
obviously you're in you're interacting with the world that doesn't exist, right,
It's it's purely digital. It is over is replacing the
physical world around you. Um, and it may mean that
you need to do things like interact with certain objects
in the world. When you include haptic feedback in that world,
(28:27):
it increases that sense of immersion that you are really
in that virtual environment. Uh, this is tricky to do, right, Like,
we don't have a whole lot of examples. I've seen
some stuff like I wore a vest at c S
one year that was a haptic feedback vest for video
games that would have lots of little actuators and motors
in it so that when you got shot in a game,
(28:49):
it would map. It would give you a good thud
at the location of the shot, so if it was
a like a shot in the shoulder, your shoulder would
actually go thud. And yeah, that's not pleasant. They also
had helmets. I did not try one of those on. Um,
being a bald guy, I'm not thrilled about putting on
helmets that other people have tried. Uh, you're the least
(29:12):
susceptible to headlines. Well, I don't want to make other
people think, oh, the sweaty bald guy just took that
helmet off. Now I gotta put it on. And I
don't like to share those kind of things. But yeah,
it's it's it's one of those things that again increases
your sense of immersion. It could be very valuable. I've
seen some really cool prototype devices that give that haptic
(29:34):
feedback to create that that more convincing situation, which can
be really useful whether it's a game or some other experience.
Like have you seen the gloves that it looks like
you put your fingers through a set of rings and
each ring is attached to what looks almost like a piston.
I'm not sure I have, so, so each finger is
(29:56):
on again they look like little pistons that that look
like almost like pneumatic pistons, right, And the idea is
that it creates resistance so that when you are picking
up a virtual object in a game. It increases resistance
as if you were actually gripping that object, so and
you could you can adjust the level of resistance so
(30:16):
a soft object has less resistance, so you can you know,
you you definitely feel it. It's more than if you
were to just open or close your hand, but a
hard object would have greater resistance, hopefully mapped so that
it would kind of like, quote unquote, feel like your
hand is closing around an object of about the same
size as what you're seeing on the screen. Now, that
(30:38):
doesn't solve the problem of the fact that everything is
weightless right like it's or or it's rather just the
weight of whatever the actual glove you're wearing happens to be. Well,
they need to make things in the gloves that adjust
their mass and suddenly so it just converts energy to
mass or releases an equivalent amount of energy when you
drop it. You know, you set off a nuclear bomb
(30:59):
every time you drop something from your inventory. No big deal, uh,
I mean, you know there are other ways of doing that,
but it mostly involves turning your game room into like
a crazy system of pulleys and restraints that are going
to raise some questions in your average household, I imagine y. Yeah,
But at any rate, the having this haptic feedback UH
(31:21):
for for augmented reality or virtual reality very important because
we are trying to marry a real experience with a
digital one to some extent, you know, uh, replacing the
real experience with a digital one in the case of
virtual reality, augmenting in the case of a R. But
either way you need to have or at least it
it has been official to have that additional haptic feedback
(31:45):
to make the experience more meaningful, powerful, memorable, uh, and
even easy to navigate. Right Like, when you have that
that force feedback and it reinforces the idea of this
is a will experience, then it is easier to to
create something that a user is going to find gratifying.
(32:09):
If you don't have that there. Ultimately you have people
who go through that experience who want to test the
boundaries of what is what it's what is possible to
do in there? And as soon as you find something
that's not possible to do, you get that sense of, oh, well,
that's disappointing. I was really hoping I could do this thing,
which is why like like there's um a game, the
(32:29):
job simulator game for VR. We talked about the previous episode.
Now there's haptic feedback doesn't really fit into this, but
the the idea that you know, you start coming up
with a concept like what would happen if I did this?
Did they think about that? And whenever you try something
and you realize, oh, they did think about it, and
(32:49):
they created a funny outcome for this thing that I
just came up with. It delights you because I mean
someone has thought the same goofy stuff you thought and
they built it into the experience. I would argue the
same thing it's true with with haptic feedback is building
the stuff in. It reminds you that other people are
(33:12):
really thinking about these weird you know, outside the boundaries cases,
and they're accounting for it. And that creates a really
fun experience on your end, a discovery that is enjoyable.
So there's not just the practical aspect of this can
help you avoid getting hurt while you're walking around a
(33:34):
physical space but you're in a virtual world. It can
also enhance that experience itself, so they're not getting hurt matters.
That doesn't matter. And we talked about this before we
came in here too, about how some of the systems
out there, like HTC vive rely on you moving around
a physical space while playing in a virtual environment. You're
(33:55):
not just sitting in a chair wearing a headset and
holding a controller. Well, that does seem like the crucial
next step. I mean, if you are inhabiting a virtual
environment in which you can move your head as if
you're really in the environment, you can't move the rest
of your body as if you're really in the environment.
I don't, I don't know. I mean, I don't have
(34:16):
much experience with virtual reality. I haven't I haven't used
any of this new generation or anything. But that seems
like that would be a kind of frustrating disconnect or
half measured. It can definitely, uh, at least in my case,
make you give you a little bit of that motion
sickness feeling. Not because usually it's that's due to latency,
(34:37):
but if if it's because you are moving about using
a controller, but you are moving your head as if
you're in the space, there's a disconnect between Hey, this
is not how I really navigate my environment in real
meat space. So my my brain is saying this is
weird because you're sitting still, you're and you don't feel
(35:00):
an emotion from the chair. You are and you're feeling
no sense of acceleration, and yet you are accelerating and
decelerating based upon the visual stimulator giving you know what.
That's it. Everyone in the stomach get out right, Like
you just start to feel kind of sick to your stomach.
Whereas with the HTC vibe and and similar products, you
(35:22):
are actually moving. You you can physically move into space,
but then you're you're you have the question of well,
how do we make sure a person moving in a
physical space doesn't slam up against the wall or trip
over a chair or bang up against a desk. As
we've talked about before, this introduces all kinds of other problems,
Like it would be even cooler to have virtual environments
(35:44):
where you could move with your body throughout the space,
but you'd have to have a dedicated real world space
it seems like, and of a decent size. Yeah, And
it would also in some way have to have to
match the virtual space, Like what if you want to
go uphill in the virtual space, Either the virtual space
would have to be completely flat, or you would have
(36:06):
to have like real world objects that are mapped to
virtual objects, which I've seen, and at that point, like
it's almost like why yeah, Because I mean like like
I've seen some examples of let's have this this post
here and we map it in the game. So it's
(36:26):
a tree. So if you were to walk up, you
could reach out and you could touch, and there would
actually be a thing there where the tree is. It
wouldn't necessarily feel like a tree that would be there.
Um that kind of thing or uh, you know. I've
also seen some interesting studies about how you can guide
a person so that it feels like they're walking down
(36:46):
a straight pathway but in reality, through visual cues, you
are you're making them curves, so they're walking in a circle,
but to them it feels like they're walking in a
straight line. That kind of stuff is interesting, but ultimately
you still have the concerned what happens when they get
too close to the wall with the HTC vibe. The
solution is to give a visual cue. Because you're using
(37:07):
two sensors called lighthouses that are able to map out
the room that you are in. You're supposed to put
them in in diagonally opposite corners, and when you get
close to a wall, there's an indication on in your
view that you're getting too close to a wall. But
you could do the same thing with haptic feedback, where
you get a little bit of a of a pressure
(37:27):
against some part of your body, not necessarily uh linked
to gameplay, because you don't want to You don't want
to create confusion, right, You don't want to make people
think was that because it thundered in the game, Or
am I I'm about to break my nose because I'm
getting too close to the wall. But if you're able
to differentiate, differentiate, and use that haptic feedback, it is
(37:48):
kind of like having that spiky sense of Oh, I
gotta I gotta be careful or I'm gonna I'm gonna
totally uh smack my face against the door any second, now,
you know. I wonder if in the in for navigating
virtual worlds, what people are going to have to have
is is not dedicated flat rooms, but a combination of
(38:08):
the omnidirectional treadmill paired at the same time with a
sort of actuated version of those coffee table art pieces
people have where they've got all the pins in them,
and you can create a three D contour of your
hand by sticking them under the pins. First time I
ever encountered that was at the Imagination Pavilion at Epcot. Really,
I thought it was the most mind breaking thing I'd
(38:30):
ever seen. I mean, they are pretty cool. I like
to play with them. Whenever I see one, I'm impressed.
But anyway, wait, where is it? I used to encounter
one all the time. I can't remember. For me, it
was the early eighties when Epcot first opened. I was
a kid, and I was easily impressed. I think one
of my friends had one. But anyway, imagine something kind
(38:51):
of like that. So you're on an omnidirectional treadmill, but
also under your feet you've got you know, actuated things
that can rise up in program contours so that you
can you can climb, you can descend, becoming like the
mini bots and uh in Big Hero six. Yeah, I
see what you're saying. Where you've got this um so,
(39:11):
so it's not just flat, featureless landscape that you're running across,
but you can you can have elevation uh and and
encounter that and have it change. Yeah, or I mean
another way of doing it would be just a tilt
like a a tilting omnidirectional uh treadmill, which we're starting
to get We're starting to get into some pretty complicated
(39:32):
territory here to have all this work out properly. But
it is interesting, um you know, and these are these
are real questions, like at what point do you just say,
all right, now I'm outside, Yeah, this is this is
too complicated, Let's just build a theme park or uh
we can we can get away with this minimum amount
(39:54):
of sensory input and create a really compelling experience because honestly,
it doesn't take that much to get lost in a
virtual experience and start to think of it as a
real experience. What you need is low latency and high agency,
Like you need to have very little lag between your
actions and you need to actually be able to enact
(40:17):
change in your environment. And the more you're able to
bring down latency and increase agency, the more immersive the
experience tends to be. And it doesn't matter how high
fidelity the graphics are at that point, or how many
different points of feedback you have, um other than you know,
as long as everything supports what you're doing and doesn't
(40:37):
detract from it, right, like you wouldn't want to have
a haptic feedback that that contradicts the other sensory uh
input that you're receiving. That would that would be detrimental
to the overall experience. But yeah, this was um a
fun thing to just explore and and I love the
idea of utilizing it in a practical sense for space application,
(41:01):
but also then to see what other potential applications we
could uh find for it here on Earth. Uh, mostly
because you know, I like to play with technology. I
don't know if you saw it, but I put on
a suit that made you'll feel like a real old person.
Yeah I I if I didn't like technology, I never
would have allowed them to strap on that extremely snug outfit.
(41:27):
So Lauren, thank you for the suggestion. Wish that you
could have been here to chime in because this was
a fun one. But we do expect her to be
back for our next episode and of course, listeners out there,
if you have any comments, questions, you have suggestions for
future topics, please send them our way our email addresses
(41:48):
f W Thinking at how Stuff Works dot com, or
drop us a line on Facebook or Twitter. On Twitter,
we are FW thinking on Facebook. You can search FW
thinking in the little search bar our profile to top
up right away and you can leave us a message
there and we will talk to you again really soon.
(42:11):
For more on this topic and the future of technology,
visit forward thinking dot Com, brought to you by Toyota.
Let's Go Places,
Brought to you by Toyota. Let's go places. Welcome to
Forward Thinking. Hey there, and welcome to Forward Thinking, the
podcast that looks at the future and says these boots
are made for walking. I'm Jonathan Strickland, and I'm Joe McCormick,
(00:21):
and our other host, Lauren voc obamb is still unto
the weather, but she will be back with us next time.
I'm quite sure. Yeah, pretty pretty pretty positive on that.
And don't don't worry about her. She's doing fine. She
just got this scratching through. It's a bit of a
shame because she's the one who actually pitched this episode topic,
wasn't she. I think so so, Jonathan, I have a
question for you. Asked me, did you remever did you
(00:42):
ever own a pair of moon boots? I I can
honestly say, with on conviction that I have never owned
a pair of moon boots. You're aware of what I'm
talking about. That not not the actual boots that astronauts wear.
I'm so you mean the style of boots known as
moon boots. Yeah. No, I I had never owned a
pair of those. I have warned things that I suspect
(01:05):
we're equivalent to moon boots, at least in their the
clunkiness factor. But I personally have never owned any Oh
that may be what I had when I was a kid.
I had some puffy boots that I referred to as
moon boots. I don't know if moon boots is a brand,
that a brand. Well anyway, I don't know one way
or another. But I had some puffy boots that were
(01:26):
quite beloved to me, and I wore them out in
the snow, and I wore them out in the summer
also just because I thought they looked cool. Wow. Okay, well,
you know it just tells me a lot about you, Joe,
and and be proud of who you are. But we
wanted to talk today about a specific type of of
approach to designing shoes for astronauts specifically, but there are
(01:51):
implications beyond that that addressed some problems that they've encountered
in the past. So let's let's talk about why we
would even have this conversation in the first place. What
is the big issue? What is the problem astronauts face.
The problem is astronauts are always too intoxicated to walk
(02:13):
around properly. No that's not true, No, not at all.
That we we are. But however, if you were to
watch certain videos, you might get that impression. Because Jonathan,
did you watch the astronauts falling over? So did I? Even?
I even have a little little note that I put
on there. Okay, So Joe decided to kind of watch
this video and then he shared the video in our
(02:36):
notes of astronauts having difficulty standing upright while walking about
on the Moon's surface. And that's really what this episode
is all about, Like, how can people design uh a
system that allows astronauts to walk around on a surface
(02:56):
like the Moon's surface without the danger constantly falling over,
because based upon the YouTube video you've sent, it looks
like that is a common issue. Well, it's kind of funny,
isn't it, Like they walking can be hard when you're
on the moon, you know, yeah, I I wrote. My
comment on the video was this is majestic. You really
(03:19):
get the sense of achievement and grandeur as you watch
chubby figures fall about in the dust. It really is
like watching little toddlers fall over in a sandbox a lot.
But it's not the fault of the astronauts. It's just
really hard to get around in bulky spacesuits. And once
you're down, it looks like it's pretty challenging to get
back up again. Yeah, they kind of try all these
(03:40):
little moves to sort of pop back up, but they
keep falling over and they look like crazy breakdancing moves
at some points. Because there's one point where you see
a guy who is holding himself up in a horizontal position,
like like parallel to the Moon's surface, on his hands
and just trying to push himself up so they can
get his feet up underneath himself. It's pretty incredible. And
(04:03):
of course this isn't just the Moon. This is presumably
true for other space environments like Mars and other foreign
planets as well, where asteroids as well. That kind of
probably even harder on asteroids, the lack of i mean
almost you know, no negligible gravity. Yeah yeah, so, uh
so space suits are bulky their pressurized environments, and they
can tend to cause a feeling of perceptual segregation from
(04:27):
the outer atmosphere. You know, you're like you're living in
this interior world and just sort of vaguely bubble buoying through. Yeah,
like your perception is thrown off. Yeah, so they have
to be bulky, of course, because if you know, they
have all these built in protections, shielding regulatory systems. But
this often leaves the astronaut with a sort of limited
(04:48):
ability to sense their surroundings both by sight and by touch.
Right and then with that reduced gravity that I would
imagine adds another element of uncertainty and and and kind
of clumsiness to the whole thing. I imagine that walking
around on the Moon is a lot like have you
ever gone up a set of stairs that you've gone
up and down a thousand times and you're not really
(05:11):
paying attention, and in your mind you're thinking there's either
one more or one fewer step than actually are there
on the stairs, And that is a a really alarming
thing to have happened to you that moment. You either
take an extra step on a stair that's not there,
or you fail to take that extra step and you
(05:33):
miss a step that actually is there. I argue often
that it's worse going down than going up, Like I've
tried to step down at the bottom of a set
of stairs when I thought there was one more to go,
and it is the most jarring experience to try and
step down where you cannot because there is solid floor there, though,
I imagine it is much more jarring to step down
thinking you should be experiencing about you know, point eight
(05:56):
more g s than you are, right, and then you
end up falling over as a result because you have
either underestimated or overestimated. You couldn't really tell when you
were making contact. You didn't really know when you could
shift your weight, and so you either do so prematurely
or you're not quite ready to do so when you
need to. All of these factors come into play, and
(06:17):
it explains why we get this wonderful sequence of astronauts
falling over as if they're just completely helpless on the
surface of the moon. But of course, when you were
on a literal moonwalk falling over, despite how funny the
video is, that's not a good idea. You don't want
to be doing that. Number One, space suits are pretty
well shielded, but there's always a chance that you could
(06:38):
damage or puncture something vital. That's just pretty much everything
on the space suit. Yeah. Then on top of that,
there's just the waste of limited time and oxygen resources
lying stuck on the ground like this marshmallow man turtle thing. Right,
If you have a specific set of goals for your moonwalk.
You don't want to get stuck for ten minutes trying
to get up. Yeah, becausebviously, whenever any of that stuff
(07:01):
gets planned out back on Earth in the initial stages
where you've you've got your mission parameter set, you're gonna
build in extra time. Right, You're never going to say,
let's let's really cut this to the wire, right that
that makes no sense. However, if you fall over, then
you start eating up that extra time that was built in,
and and this is a real concern. Yeah, So the
(07:22):
story that made us want to do this episode today
is about a particular response to this kind of problem.
So the story refers to some work done by M I. T. S.
Department of Aeronautics and Astronautics or Aero Astro and the
Charles Stark Draper Lab in Cambridge, Massachusetts. I love the
fact that's in Stark Labs. Yes it is, but they're
(07:46):
not that Stark because he works alone. It's a Stark contrast,
I imagine it is. Now. So they've been working on
this potential solution and it is space boots with environmental
swerves and embedded haptic motors. So it works like this.
Imagine you are an astronaut on the surface of the
(08:07):
Moon and you're shoveling up some barrels of lunar regular
to transport back to Earth, where it has become a
wildly popular flavoring agent in hot cuisine, of course. But
of course you are walking through a shallow dip on
the lunar surface and you feel a sudden vibration on
the outside toe of your left foot, so you stop,
you take a step back, You look down, you see, oh,
(08:28):
there it is. It's a rock, or maybe it's a
little room bus sized robotic rover sitting deactivated in your way,
whatever it is. Having avoided tripping over it, you go
around scoop up some more of the ultra fine particles
of cash money. Pretty simple, right, So you don't have
to depend on your eyes. You've got a tactle sensation,
(08:48):
this vibration that warns you to stop moving and reach
out your course. And that's exactly what we mean when
we talk about haptic feedback. If you're not familiar with
the term haptics, it really means engaging your sense of
touch in a way that gives you more information about something,
whether it's your environment, or a virtual environment or something
(09:10):
along those lines. A very basic example of this is
you likely have a smartphone or cell phone that has
a little motor that vibrates when you get a call
or a text. That's a haptic feedback system telling you, hey,
you've got an alert. So maybe it doesn't make an
audible alert so that you know if you're at work,
(09:31):
you're not having your phone go grouper or whatever. Well,
here's one. If you have a touch screen keypad on
your phone, very likely the phone vibrates ever so gently
when you correctly press a key, so that so that
you know you did hit the key instead of sort
of going off the side and not registering the the
input right, or that you just start typing you know,
(09:53):
h h h h h h h for hello, because
you haven't figured out if you're actually, you know, making
that contact with the screen or not right. And I
found out when when my phone's power saving features go
on when it's low on battery and it stops doing
that that response. Yeah, I find it disturbing when I'm
typing with my fingers, I'm like, wait a minute, am
(10:15):
I am I typing? I don't know if the letters
are going Yeah, I've gotten used to it. I had
turned mine off when I was at home, um after
a certain time, Like I could, I could create parameters
of when haptic feedback was turned on or turned off,
and that was specifically so that I didn't disturb either
my wife or my dog whenever I decided to look
(10:36):
on something on my phone and look something gup, because
every every time I would type, you get this little
haptic buzz and it would be distracting. But I also
found it very difficult to navigate on my phone without
it because without that feedback to reassure me that yes,
indeed your touch has been has been registered, I felt
(10:57):
like I wasn't properly activating my phone. So haptic is
a very powerful kind of disciplines of a very powerful
technology or strategy. I guess you could say to engage
our senses in another way and give us additional information. Yeah,
it's a it's a way of repurposing information that you
would get through one sense and feeding it through a
(11:19):
different one. So in July of this year, researchers presented
their findings on on these haptic feedback boots boots for
space at the International Conference on Human Computer interaction, and
specifically they've been studying what types of sensations applied to
which regions of the foot do the best job of
(11:41):
keeping people from tripping over things, or actually that that
part would be inferred, because really what they're looking for
first is like when people can feel the sensation and
how well they can feel it right, and and can
they identify from what direction it's it is coming? So
that way, like if we were to have the outer
left edge of your foot, uh, buzz, would that translate
(12:05):
in your mind, Oh, there's something forward and to the
left of me that I need to look out for. Yeah, uh,
And and there are lots of different options for how
you could provide haptic feature right. So you know, our
nervous systems are kind of quirky for reasons of evolutionary happenstance,
and so they respond better to some stimuli than to others.
So they actually looked into what works best. So scientists
(12:29):
Alison Gibson, Andrea Webb and Leah Sterling conducted this study
to find out exactly how good people are at detecting
different kinds of sensations on their feet under various conditions.
On my feet, my feet were tingling reading about this. Yeah,
and it's interesting to hear about the experiments because they
included everything from kind of a straightforward approach where they said,
(12:50):
are you gonna feel a buzz? I need you to
tell me, like, on a scale of maybe one to ten,
how strong was that buzz and where on your foot
did you feel it? And then they start and making
it a little more complicated by making people like do
things while they're administering the buzzes and try and see like, well,
can they still accurately describe the intensity and the location
(13:12):
because this obviously that would be important. You know, if
you're walking around the surface of the moon, you're probably
not just completely focused on how are my feet feeling
right now? No, you're trying to get that delicious regulars
back to your back to your transfer. Emerald will not
wait for his for his moon spice bam. So if
you want to look up their their presentation, it's it's
(13:33):
called user abilities in detecting vibro tactle signals on the
feet under varying attention loads. Vibro Tactle is amazing, what
a great word, And yeah, varying attention loads obviously that's
the reference to what I was talking about earlier. But
you know, sometimes you have to create some distractions, nothing
of real like importance or or urgency, but rather just
(13:57):
something to occupy the person's mind. So that example is
one of the independent variables they tested attention state whether
you're distracted by other stuff. Another one was vibration locations
of tow or heel in step or outside, etcetera. And
then the third was vibration signal type. And this is
not correct, but I wish this meant good vibrations versus
(14:19):
bad vibrations bad vibes manuh no. In actuality, the different
vibration types were high, low, increasing and decreasing, increasing and decreasing.
Matter because of something I'll get to in a bit, well,
well before before we get into that, just for a
clarification on my own part, because this was something that
I missed when I was reading it. When they say
(14:39):
high and low, are they talking about frequency or intensity? Oh? Well,
actually I think it's intensity, But I'm not positive because
I was it's intensity. I would think so too, because
to me that would just mean like, is the motor
vibrating at a greater intense stealer as in, is it
making heart of great bigger vibrations is the ample alitude
(15:00):
increased as opposed to is the frequency increase. I would
imagine the same thing. That was my interpretation, but I
didn't actually read Yeah, okay, co uh So these could
be delivered to six different places on each foot and
that results in twenty four unique different kinds of signals.
And then they also, like you said, they ran a
(15:20):
distraction test, so they were like, let me know if
you can feel anything on your feet and uh and
what it is, where it's coming in, and what it
feels like. But then also do that while I show
you a random number and you start to count up
by threes Jonathan twenty four, thirty thirty three, thirty six.
I gave you an easy one. Let me get a
hard no no, no, no no, don't don't have a
(15:43):
liberal arts major. Uh So, But it turns out even
with a pretty simple test like this subject has had
some difficulty. They they had trouble identifying when the vibration,
for example, was steadily increasing, so that that's not just
the on and off high and low sensation, but when
it's slowly getting more intense and even in the undistracted condition,
(16:06):
so even when they're not trying to do the counting.
People did not do a great job of detecting when
vibration was decreasing in intensity. Uh. And also no surprise here,
but some parts of the foot are just more sensitive
than others. You know, the middle of the outside of
the foot was pretty dumb. By the way, A little side. Now,
do you ever play that game where you touch somebody's
(16:27):
back with different numbers of fingers and asked them how
many it is you should you should agree beforehand with
the person you're playing, right, there's consent is absolutely don't
don't surprise somebody with this game. I've I've played variations
of that, like as a kid. There's also the I'm
gonna draw something on your back, tell me what it
was you I drew and I, uh, it's difficult. Yeah,
(16:50):
people don't realize how dumb their backs are. The tactle
neurons back there is just I don't know, you know,
I'm touching your touch somebody with three fingers and they're like,
oh one right, yeah, let's say. And so you think
about it, like, there are definitely gonna be areas of
the foot that will be more responsive to to haptic
feedback than others. And when they were talking about increasing
(17:13):
or decreasing the intensity, I could easily imagine that being
a difficult thing to uh, to detect even so much.
Is like, even if I have a phone in my
hand and I'm increasing the motor intensity until you get
to like unless it unless it does does it in
big jumps, the differences can be fairly subtle, And then
(17:35):
I think, well, what if I felt that on my foot,
and what if that were the important determining factor. Like,
as I approach a potential stumbling block, the intensity increases. Uh,
if it is to settle, then that becomes kind of useless.
It just is going to be a constant distraction as
I look around, and maybe I can't identify what the
(17:56):
thing is because it's not quite close enough for me
to notice that it is something I'm going to stumble over. Uh.
And then next thing, you know, I start to ignore
this this buzzing because so frequently it's not helpful, and
then I'm falling all over again. That exactly is a
problem people are worried about. Okay, so how do you
keep people from tuning out the feedback? Yeah, you have
(18:17):
to have it be urgent enough that it's meaningful, but
not so urgent that it is that it's misleading right there.
It's a delicate balance, but you can see where it
could become really useful if you have the sensors in
a boot and you have the alert happen where it's
(18:37):
not gonna be so sensitive that it's it's it's set
off by everything, because then you're just like, well, all
I can tell you is being on the moon is
crappy because all my feet are just buzzing. These dogs
are buzzing. In other words, that's an alternate title for
this episode. Yes, so of course this technology it's still
(18:58):
got a lot of ground to cover as the study,
but a lot of moon ground. Because while a simple
on off vibration might be somewhat helpful, one thing about
the world around us is it's best represented by continuous data,
not on off analogue rather than digital. Yeah, so the
the increasing and decreasing quantities would probably be the best
(19:19):
way to represent something like proximity or even like the
the size of the potential obstacle, because again we have
you have such a limited field of view inside a
space helmet, right like it cuts off your field of
view significantly. Uh, you might want to use intensity to
indicate perhaps not the distance to the obstacle, but perhaps
(19:42):
the the size of the obstacle, like whether or not
it's a sizeable rock that's ahead of you that you
might stumble over, or a drop down that's been you know,
that's your boot has uh detected where you know you
don't want to have that next step forward be four
inches lower than what you had anticipated. Yeah, exactly. So
(20:03):
the mn T news piece that covered this particular research also,
I thought this was interesting. Spoke to a doctor, Shirley
right Dick, professor of Health and Kinesiology at Perdue, who
specializes She specializes in like falling down. Basically, so do I,
but not not. I don't have the credentials. I just
have the experience America's Funniest Home Videos professor. And but
(20:28):
she pointed out that this would be useful for more
than just astronauts. I think that that's clear. But for who,
Well she gave the example of how about firefighters. So
for firefighters, you go into a burning building, there's flames,
there's smoke. Environmental navigation inside a burning building is a
well known problem. Uh, And so this could be very
useful for them too, And I imagine maybe deep sea divers.
(20:51):
Maybe you know there there are lots of conditions where
you don't want to be tripping over stuff and your
personal visibility is going to be limited. And the even
beyond that, we started talking and in fact articles have
also been pointing this out. It's not just Joe and
I have come up with this idea, but we've talked
about how these sort of haptic feedback systems could be
(21:11):
used for people in everyday situations, particularly people who may
have uh problems with their vision. Yeah, exactly, So one
obvious application would be for people with visual impairments. So
at a certain level, this is harder than it sounds,
because you can give somebody a vibration in their shoe
(21:31):
if they're about to kick something, But how do you
communicate more complex navigational information about a person surroundings without
visual information. The thing about the astronauts is they should
be able to see generally what's going on around them.
The problem is they might not notice there's a rock
at their foot. But but imagine you're completely blind and
(21:52):
you're trying to use haptic feedback to give people navigational
information about what's around them. You can you can give
a kind of dumb and form, but it gets more
difficult the more info you're trying to encode, right, So,
so indicating that there's a possible surface that you're going
to come into contact with right there, that's that's one thing.
(22:14):
But to get to a level like the way, how
about there's a street and there are cars on it
right right. One of the things I thought about when
we started talking about this is it's a a step
toward spiky sense, but perhaps a better representation would even
be Daredevil's heightened senses. That Daredevil's capable even though he's blind.
(22:34):
Matt Murdoch spoiler alert is blind, he's capable of sensing
through this this crazy heightened ability to almost see vibrations
like he can he can sense sound at such a
level that he can and he knows shapes of things
and whether something is coming towards him or moving away
from him. That level is well beyond a little motor
(22:59):
vibrating against you to indicate that there's something you might
be bumping into in the very near future. And um
and while I think that this technology definitely has the
potential to increase a person's environmental awareness, particularly someone who
perhaps has has just become blind, who hasn't had hasn't
(23:21):
been dealing with an impairment over the course of years. Uh,
this might be something that is more helpful. Yeah, and
this is something that is also under development. In fact,
more m I T research. There. There's research on the
hardware behind a wearable device that uses three D camera
data to provide navigational feedback to people with visual impairments.
(23:43):
It's been known as the Virtual Guide Dog. Interesting. But
you know, Jonathan and I were talking before we came
in here about the idea of of expanding this even
farther with something that Jonathan dubbed pedestrian assist. Yeah, kind
modeled after the driver assist systems that were familiar with
(24:03):
things like you know, the lane assist, parking assist, breaking assist, acceleration,
you know, cruise control. But in this case, obviously I'm
not talking about your your shoes magically taking over for you, Like, hey, shoes,
I want to go to the mall and they just
start stepping and you're just like, hey, I can go
to sleep now. Yeah, So it would be self powered,
but it would be uh, sort of like a not
(24:24):
self driving car for your feet, but self navigating car
for your feet or or at least giving you Yeah,
that driver assists that warning accident prevention right like you're
going to hey, buddy, stop playing Pokemon Go for three
freaking seconds. That is a thing, because you're gonna run
into that street lamp. People have walked off of ledges,
haven't they? I I have personally like I I play
(24:46):
Pokemon Go, so full full disclosure. I play the game.
I do not disclosing your conflict of interest, Yes, exactly, well,
I want to make sure that people know that I'm
not making fun of folks who play Pokemon Go. I'm
making fun of myself, a person who does play Pokemon Go.
So subtle difference. I have made it a point when
I started playing, and i've I've stayed true to this
(25:09):
that I will not pay any attention to my phone
whenever I am crossing streets, because I know I would
be that person who stumbles over a curb and causes
danger to myself, to others, to you know, giving drivers
a heart attack, maybe possibly suffering a massive injury or
perhaps even death because I'm not paying attention to what
(25:31):
I'm doing, Because I am aware of this, and I've
had situations where I have made contact with stationary objects
because I was looking down at my screen and not up.
Now in in my case, that stationary object was a
tree branch that in my neighborhood. There's this one tree
where the branches is slightly uh lower down to the
(25:53):
ground than my height is. So if I don't duck,
I will bang my head against it. But if I
if I'm looking down towards my screen, it's out of
my view, my field of view. So I have hit
on more than one occasion the top of my head
against this branch. Now, it hasn't always been because of Pokemon.
Sometimes it's a real monster, that being my dog. So
(26:16):
I'm often looking down at him, and I frequently nearly
not quite close line myself. It's not quite low enough,
but I do get knocked right at the crown of
my head. Oh, I thought you were going to say
that you were on the way home and face timing
with him before. If my dog learns how to face time,
it will be the end of times. But because I
will never do anything ever again other than FaceTime with
(26:38):
my dog. But no, it is definitely one of those things.
Where if I were to have some sort of of
sensor on me and clothing, they could say hey, dude,
heads up, then I would not have that experience. Now,
in my case, that was a very minor inconvenience. I
was never hitting it so hard that I was causing
myself real injury. It was more more than anything else.
(27:02):
I was feeling embarrassed about clocking my head because I
wasn't really paying attention to where I was going. But
this could really end up being something that prevents injury
and death if if actually rolled out to and incorporated
in a larger number of of wearables. Yeah. And of course, Jonathan,
you were mentioning something about the way haptics would be
(27:24):
incorporated in uh in virtual reality and augmented reality environment. Yeah,
so this is not unusual either. It's something that a
lot of In fact, I would argue that the video
game world has pushed the haptic discipline forward more than
anything else as an effort to create more immersive experiences
for gamers. So a very simple version of that is
(27:46):
the rumble pack that you find in a lot of
game controllers, right, that gives you that So like like
when there's thunder in the game, the controller vibrates and
gives you that sensation something that you don't even really
necessarily register after you've played a lot of games with
that kind of functionality. But when it's gone, you're like, oh, yeah,
that is that does make a difference. Well, in virtual reality,
(28:07):
obviously you're in you're interacting with the world that doesn't exist, right,
It's it's purely digital. It is over is replacing the
physical world around you. Um, and it may mean that
you need to do things like interact with certain objects
in the world. When you include haptic feedback in that world,
(28:27):
it increases that sense of immersion that you are really
in that virtual environment. Uh, this is tricky to do, right, Like,
we don't have a whole lot of examples. I've seen
some stuff like I wore a vest at c S
one year that was a haptic feedback vest for video
games that would have lots of little actuators and motors
in it so that when you got shot in a game,
(28:49):
it would map. It would give you a good thud
at the location of the shot, so if it was
a like a shot in the shoulder, your shoulder would
actually go thud. And yeah, that's not pleasant. They also
had helmets. I did not try one of those on. Um,
being a bald guy, I'm not thrilled about putting on
helmets that other people have tried. Uh, you're the least
(29:12):
susceptible to headlines. Well, I don't want to make other
people think, oh, the sweaty bald guy just took that
helmet off. Now I gotta put it on. And I
don't like to share those kind of things. But yeah,
it's it's it's one of those things that again increases
your sense of immersion. It could be very valuable. I've
seen some really cool prototype devices that give that haptic
(29:34):
feedback to create that that more convincing situation, which can
be really useful whether it's a game or some other experience.
Like have you seen the gloves that it looks like
you put your fingers through a set of rings and
each ring is attached to what looks almost like a piston.
I'm not sure I have, so, so each finger is
(29:56):
on again they look like little pistons that that look
like almost like pneumatic pistons, right, And the idea is
that it creates resistance so that when you are picking
up a virtual object in a game. It increases resistance
as if you were actually gripping that object, so and
you could you can adjust the level of resistance so
(30:16):
a soft object has less resistance, so you can you know,
you you definitely feel it. It's more than if you
were to just open or close your hand, but a
hard object would have greater resistance, hopefully mapped so that
it would kind of like, quote unquote, feel like your
hand is closing around an object of about the same
size as what you're seeing on the screen. Now, that
(30:38):
doesn't solve the problem of the fact that everything is
weightless right like it's or or it's rather just the
weight of whatever the actual glove you're wearing happens to be. Well,
they need to make things in the gloves that adjust
their mass and suddenly so it just converts energy to
mass or releases an equivalent amount of energy when you
drop it. You know, you set off a nuclear bomb
(30:59):
every time you drop something from your inventory. No big deal, uh,
I mean, you know there are other ways of doing that,
but it mostly involves turning your game room into like
a crazy system of pulleys and restraints that are going
to raise some questions in your average household, I imagine y. Yeah,
But at any rate, the having this haptic feedback UH
(31:21):
for for augmented reality or virtual reality very important because
we are trying to marry a real experience with a
digital one to some extent, you know, uh, replacing the
real experience with a digital one in the case of
virtual reality, augmenting in the case of a R. But
either way you need to have or at least it
it has been official to have that additional haptic feedback
(31:45):
to make the experience more meaningful, powerful, memorable, uh, and
even easy to navigate. Right Like, when you have that
that force feedback and it reinforces the idea of this
is a will experience, then it is easier to to
create something that a user is going to find gratifying.
(32:09):
If you don't have that there. Ultimately you have people
who go through that experience who want to test the
boundaries of what is what it's what is possible to
do in there? And as soon as you find something
that's not possible to do, you get that sense of, oh, well,
that's disappointing. I was really hoping I could do this thing,
which is why like like there's um a game, the
(32:29):
job simulator game for VR. We talked about the previous episode.
Now there's haptic feedback doesn't really fit into this, but
the the idea that you know, you start coming up
with a concept like what would happen if I did this?
Did they think about that? And whenever you try something
and you realize, oh, they did think about it, and
(32:49):
they created a funny outcome for this thing that I
just came up with. It delights you because I mean
someone has thought the same goofy stuff you thought and
they built it into the experience. I would argue the
same thing it's true with with haptic feedback is building
the stuff in. It reminds you that other people are
(33:12):
really thinking about these weird you know, outside the boundaries cases,
and they're accounting for it. And that creates a really
fun experience on your end, a discovery that is enjoyable.
So there's not just the practical aspect of this can
help you avoid getting hurt while you're walking around a
(33:34):
physical space but you're in a virtual world. It can
also enhance that experience itself, so they're not getting hurt matters.
That doesn't matter. And we talked about this before we
came in here too, about how some of the systems
out there, like HTC vive rely on you moving around
a physical space while playing in a virtual environment. You're
(33:55):
not just sitting in a chair wearing a headset and
holding a controller. Well, that does seem like the crucial
next step. I mean, if you are inhabiting a virtual
environment in which you can move your head as if
you're really in the environment, you can't move the rest
of your body as if you're really in the environment.
I don't, I don't know. I mean, I don't have
(34:16):
much experience with virtual reality. I haven't I haven't used
any of this new generation or anything. But that seems
like that would be a kind of frustrating disconnect or
half measured. It can definitely, uh, at least in my case,
make you give you a little bit of that motion
sickness feeling. Not because usually it's that's due to latency,
(34:37):
but if if it's because you are moving about using
a controller, but you are moving your head as if
you're in the space, there's a disconnect between Hey, this
is not how I really navigate my environment in real
meat space. So my my brain is saying this is
weird because you're sitting still, you're and you don't feel
(35:00):
an emotion from the chair. You are and you're feeling
no sense of acceleration, and yet you are accelerating and
decelerating based upon the visual stimulator giving you know what.
That's it. Everyone in the stomach get out right, Like
you just start to feel kind of sick to your stomach.
Whereas with the HTC vibe and and similar products, you
(35:22):
are actually moving. You you can physically move into space,
but then you're you're you have the question of well,
how do we make sure a person moving in a
physical space doesn't slam up against the wall or trip
over a chair or bang up against a desk. As
we've talked about before, this introduces all kinds of other problems,
Like it would be even cooler to have virtual environments
(35:44):
where you could move with your body throughout the space,
but you'd have to have a dedicated real world space
it seems like, and of a decent size. Yeah, And
it would also in some way have to have to
match the virtual space, Like what if you want to
go uphill in the virtual space, Either the virtual space
would have to be completely flat, or you would have
(36:06):
to have like real world objects that are mapped to
virtual objects, which I've seen, and at that point, like
it's almost like why yeah, Because I mean like like
I've seen some examples of let's have this this post
here and we map it in the game. So it's
(36:26):
a tree. So if you were to walk up, you
could reach out and you could touch, and there would
actually be a thing there where the tree is. It
wouldn't necessarily feel like a tree that would be there.
Um that kind of thing or uh, you know. I've
also seen some interesting studies about how you can guide
a person so that it feels like they're walking down
(36:46):
a straight pathway but in reality, through visual cues, you
are you're making them curves, so they're walking in a circle,
but to them it feels like they're walking in a
straight line. That kind of stuff is interesting, but ultimately
you still have the concerned what happens when they get
too close to the wall with the HTC vibe. The
solution is to give a visual cue. Because you're using
(37:07):
two sensors called lighthouses that are able to map out
the room that you are in. You're supposed to put
them in in diagonally opposite corners, and when you get
close to a wall, there's an indication on in your
view that you're getting too close to a wall. But
you could do the same thing with haptic feedback, where
you get a little bit of a of a pressure
(37:27):
against some part of your body, not necessarily uh linked
to gameplay, because you don't want to You don't want
to create confusion, right, You don't want to make people
think was that because it thundered in the game, Or
am I I'm about to break my nose because I'm
getting too close to the wall. But if you're able
to differentiate, differentiate, and use that haptic feedback, it is
(37:48):
kind of like having that spiky sense of Oh, I
gotta I gotta be careful or I'm gonna I'm gonna
totally uh smack my face against the door any second, now,
you know. I wonder if in the in for navigating
virtual worlds, what people are going to have to have
is is not dedicated flat rooms, but a combination of
(38:08):
the omnidirectional treadmill paired at the same time with a
sort of actuated version of those coffee table art pieces
people have where they've got all the pins in them,
and you can create a three D contour of your
hand by sticking them under the pins. First time I
ever encountered that was at the Imagination Pavilion at Epcot. Really,
I thought it was the most mind breaking thing I'd
(38:30):
ever seen. I mean, they are pretty cool. I like
to play with them. Whenever I see one, I'm impressed.
But anyway, wait, where is it? I used to encounter
one all the time. I can't remember. For me, it
was the early eighties when Epcot first opened. I was
a kid, and I was easily impressed. I think one
of my friends had one. But anyway, imagine something kind
(38:51):
of like that. So you're on an omnidirectional treadmill, but
also under your feet you've got you know, actuated things
that can rise up in program contours so that you
can you can climb, you can descend, becoming like the
mini bots and uh in Big Hero six. Yeah, I
see what you're saying. Where you've got this um so,
(39:11):
so it's not just flat, featureless landscape that you're running across,
but you can you can have elevation uh and and
encounter that and have it change. Yeah, or I mean
another way of doing it would be just a tilt
like a a tilting omnidirectional uh treadmill, which we're starting
to get We're starting to get into some pretty complicated
(39:32):
territory here to have all this work out properly. But
it is interesting, um you know, and these are these
are real questions, like at what point do you just say,
all right, now I'm outside, Yeah, this is this is
too complicated, Let's just build a theme park or uh
we can we can get away with this minimum amount
(39:54):
of sensory input and create a really compelling experience because honestly,
it doesn't take that much to get lost in a
virtual experience and start to think of it as a
real experience. What you need is low latency and high agency,
Like you need to have very little lag between your
actions and you need to actually be able to enact
(40:17):
change in your environment. And the more you're able to
bring down latency and increase agency, the more immersive the
experience tends to be. And it doesn't matter how high
fidelity the graphics are at that point, or how many
different points of feedback you have, um other than you know,
as long as everything supports what you're doing and doesn't
(40:37):
detract from it, right, like you wouldn't want to have
a haptic feedback that that contradicts the other sensory uh
input that you're receiving. That would that would be detrimental
to the overall experience. But yeah, this was um a
fun thing to just explore and and I love the
idea of utilizing it in a practical sense for space application,
(41:01):
but also then to see what other potential applications we
could uh find for it here on Earth. Uh, mostly
because you know, I like to play with technology. I
don't know if you saw it, but I put on
a suit that made you'll feel like a real old person.
Yeah I I if I didn't like technology, I never
would have allowed them to strap on that extremely snug outfit.
(41:27):
So Lauren, thank you for the suggestion. Wish that you
could have been here to chime in because this was
a fun one. But we do expect her to be
back for our next episode and of course, listeners out there,
if you have any comments, questions, you have suggestions for
future topics, please send them our way our email addresses
(41:48):
f W Thinking at how Stuff Works dot com, or
drop us a line on Facebook or Twitter. On Twitter,
we are FW thinking on Facebook. You can search FW
thinking in the little search bar our profile to top
up right away and you can leave us a message
there and we will talk to you again really soon.
(42:11):
For more on this topic and the future of technology,
visit forward thinking dot Com, brought to you by Toyota.
Let's Go Places,
Fw:Thinking News
Hosts And Creators
Jonathan Strickland
Joe McCormick
Lauren Vogelbaum
Popular Podcasts
Las Culturistas with Matt Rogers and Bowen Yang
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.