January 30, 2024 • 46 mins
Enjoying the show? Hating the show? Want to let us know either way? Text us!
In the second of their two part series on AR, Rob and PJ discuss the Apple Vision Pro, the use case displayed thus far, and whether this thing has legs in the marketplace.
They also explore the social (or more precisely, anti-social) aspects of the AR, VR, XR space and question whether this is the right direction to be going in.
Finally, they also ask the question...how big a resolution do you actually need before it really looks "real world"?
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
PJ (00:01):
Hi folks.
This is the second of a two-partseries on augmented reality.
If you haven't listened to thefirst, you might want to go back
and listen to that one.
Just to get a little groundwork.
On the other hand, maybe youjust want to jump to the vision
pro and talking about augmentedreality on the apps and the
product side of things.
(00:22):
That's okay too.
But regardless, we wanted togive you a fair warning.
And so now the conclusion.
Of augmented reality.
Ierengaym.
com ierengaym.
(00:43):
com
It's really interesting for you to have connected wanting to
take movie special effects,which as you put it, come across
perfectly.
But you know, one of the hitcheswith that is that all movies are
(01:04):
basically all 2D images, right?
So we do this, this sort ofperfect combination, but it
ultimately slams down to a 2Dimage.
And I felt that was aninteresting connection point to
take us right back to mixedreality at this point in time.
So we've done vr, we've done ar.
(01:25):
Now let's talk about the magicthat is the Vision Pro or what
it's actually doing.
Rob (01:31):
The Vision Pro, it was announced for developers last
Friday, the 19th of January, Ibelieve it was.
And it'll be for sale on, or atleast for order on the 3rd of
February.
And then ultimately people willget these in the hands and we'll
start to get reviews of them andall the issues potentially exist
(01:56):
start to come up.
It's gonna be very interestingto see how Apple, with their
fully self-contained ecosystem.
And how is this gonna interactwith everything else?
And I've seen a lot of this, Ihaven't seen it in a couple
years, but, it's definitelygonna integrate with the
ecosystem.
So.
(02:16):
That brings us to the questionof what is the vision pro?
Where did it come from?
And there's always been rumorsabout Apple making glasses, and
they're just gonna be a pair ofsuper thin framed, ultra
stylistic glasses that you canwear.
And as we've talked about, it'snot actually possible even for
Apple because there's so muchhardware that has to be in the
(02:39):
head space and it will betethered.
You did make a thin pair ofglasses'cause it would need to
be powered.
There'd be no room for batteriesor things like that but the
Vision Pro isn't a pair ofglasses as we've all seen.
It's, it's a headset.
It's technically a VR headset ithas no visible light from the
outside world into the actualdevice and in ultimately into
(03:02):
your eyes.
In fact, it has a very goodlight seal around the device to
stop outside light getting in.
So why are we calling it a mixedreality device when all in
intent and purpose, it's, it's aVR device and the reason is
cameras to be specific.
14 cameras on that device.
(03:23):
That's a awful amount ofbandwidth to handle all the
time.
That's why it has customsilicon.
Those cameras are dedicated toall different jobs.
Some of them face out, some ofthem face down to look at your
hands, some face looking at youreyes.
Uh, some are infrared.
There's a depth camera.
(03:43):
There's many forms of cameraslooking all directions,
different frameworks, differentresolutions.
So.
What they're calling mixedreality or XR or whatever other
name they want to give.
It is effectively going back towhat I said about video and
video playback on the Oculus ina standard VR device in real
(04:03):
time.
So we're taking live feeds fromcameras, projecting it onto the
displays, and so it looks likeyou're not wearing a headset.
You could see the real worldand.
It's more than that becausethat's just pass through camera.
Oculus played with this.
There's various things that theyplayed with years ago that
(04:25):
people 3D printed things and putcameras on.
There's a whole bunch ofproblems to solve here with
projection and making sure thatit looks the same as it would
look if you were to.
Not have the headset on.
So you've gotta undo thedistortions on the lenses, on
the cameras and processor canput it back into your eyes.
(04:47):
Now obviously there is latencythere, there is a little bit of
latency in that process.
How fast can you do that?
In reality?
Very fast.
'cause you really taking a framefrom one thing, adding it to a
display and that can be veryquick.
In fact, apple have a patternmode.
If it crashes, it willautomatically, instantly
Hardware.
Pass through so you're not stuckwith this device on your head
(05:10):
that you can't actually see outof.
So it's one thing that no otherdevice has.
Like Oculus, if it crashes, youend up with a crash still image
on your face and you've gottatake the headset off.
And in the process you mightfall over.
Whereas Apple have this watchdogsystem where if you don't
respond to the timer quickenough, it will automatically in
(05:31):
hardware go at this point.
Instantaneous pass through mode.
So it just looks like it's apair of glasses at that point.
'cause you're looking at thereal world, but you're looking
through a pair of cameras,you're not looking at the real
world.
Like I said, there's no way forlight to get in this device.
PJ (05:46):
Earlier, Rob, we talked about 10 seconds as being that,
you know, 10 seconds are underas being kind of that space you
want it to be.
Maybe 16, sorry, not seconds.
16.
10.
16 milliseconds to avoid theseasickness.
So I take it that with the passthrough mode, we're well
underneath that at that point in
Rob (06:05):
Oh yeah.
It's, it's instantaneous.
it's it's scanned one and scanthe other.
It's just like boom, like superlow agency.
And so that's one thing it hasand.
In all intended purpose, it doesall the things a VR device does.
It's very integrated with theApple ecosystem.
Uh, it'll have iMessage support,it plays iPad apps in 2D space
(06:30):
and all that.
It has a head start over allcompeting devices, and it has
Apple's name behind it, but itstill has all the same problems.
For example, if you open a webbrowser, how high resolution
does the display have to bebefore you can read text on a
window that's six feet away fromyou?
And that's two 4K displays veryhigh resolution, but nowhere
(06:54):
near high enough resolution.
To be able to read text at adistance, just put your laptop
screen and walk across the otherside of the room and see if you
can read it.
And that's actually a high riskscreen at a distance.
Now, make that a low risk screeneffectively and you can't read
the text.
And text is incredibly difficultto handle in a VR device.
(07:14):
All of the devices, up untilnow, magic Leap and Oculus all
had this same problem we used tohaving a.
Laptop screen, which ispotentially higher than 4K,
right?
In a phase, it's a foot away.
Now you've got that entireresolution.
Rather than being a tiny littlesolid angle of your actual
vision, it's now the wholething.
(07:36):
And which pro has a very widefield of view.
So if you factor out like howmany pixels per degree, that
display needs to be like fivetimes the resolution before you
can reliably handle text at.
Where you'd expect to be able toread it.
Obviously, if you put a a letterpiece of paper a hundred feet
away, you're not gonna be ableto read it in the real world
(07:57):
either.
So resolution at that scalebecomes less relevant, but five,
10 feet away with a a bigvirtual desktop, you, you're
gonna be wanting to be able toread the text and Apple do a
very good job of it, but it'sstill gonna have some of these
classic artifacts.
That the other VR devices havewhen it comes to small objects
(08:20):
door effect is minimized, butit's still there if you look for
it.
That passthrough mode that wejust talked about is how that,
in the keynote day, said thatsomeone, if you are watching a,
a movie in full immersive mode,someone walked in the room.
It'll blend into the real worldso you can kind of see what's
going on in the real worldwithout taking the headset off.
(08:43):
That same pastoral system is howit's doing that.
It's just blending Canberraimages with virtual images and
giving you that, uh, a a viewingto the real world.
But, but again, that's stillkind of vr.
What's mixed reality?
Well, mixed reality is, well,now we have this.
Synthetic vision.
(09:04):
Basically we have these camerasthat are feeding in.
We can render effectivelyaugmented reality style on top
of those camera images anddisplay that in your face.
And visibly it looks just kindalike ar.
But in a lot of ways it's a loteasier because first of all, you
(09:24):
can manage latency a lot easier.
In true ar.
The real world is the realworld.
It's gonna do what it does.
In mixed reality, you can delaythe frames from the camera up
until the latency limits thatare toleratable for VR
basically.
So if you take a frame from acamera that can be timestamped,
(09:49):
this is exactly the time thatframe was captured.
You can render at exactly thattimestamp, so there's no
shimmying around in the scene.
It's not like the virtualobjects are gonna move with
respect to the captured framebecause you know the exact
timestamps and you can renderall that, process it all, and
then display it 10 milliseconds,20 milliseconds later, and the
(10:12):
frame looks complete.
So the whole virtual objectsskied around due to the real
world not having latency and thevirtual object having latency.
It's fixed.
The whole thing now has latencyin the same way that VR does and
PJ (10:28):
so to, put a fine point on this, we are cheating in the
sense that we're delaying thereal world, you know, 10, 20
milliseconds.
And that's okay because that'sstill within the tolerance of
seasickness, but we get to usethat to synchronize the virtual
and the real feed together.
Rob (10:50):
yeah.
Basically we have a stream offrames coming in from the real
world.
None of them are visible in realtime.
They're all delayed slightly,and that delay allows us to
render on top of them.
And we still have to doeverything we have to do for
real.
Ar We have to render, take twocamera images, render stereo on
top of those.
They get projected in the glass.
And now we, you've got thiscamera based reality with
(11:13):
virtual objects in it.
So the latency is a lot easierto manage.
You still have to do the headtracking, head prediction and
all that just to get the numberdown low enough.
But, uh, the rendering side onthe, just the purely technical
side of this, generating animage is significantly better
because you can have, I knowthis frame needs to have a 3D
(11:37):
image at this timestamp, and Ican blend them perfectly.
You still have all the existingproblems that we talked about
with true ar.
Just because you are in cameraspace instead of real world
light space doesn't mean theocclusion problem goes away,
depth clipping and all thingslike that still needs to occur
PJ (12:00):
The shadow problem's easier though.
Rob (12:03):
the shadow problem is now fixable because.
The image is now virtual.
The camera image is just a pixelbased image.
It's not the real world.
So we can subtract light fromit, we can replace pixels with
black, and if we have a fineenough understanding, then
lighting becomes much easierbecause we can do blended light,
(12:23):
we can do subtractive light.
It's not just additive like itwas in the true AR sense.
So visual effects get better.
Dynamic range gets better'cause.
We're not just adding lighttowards, it's already there and
you still have all the sameproblems of if a light's
flashing in the distance andyou've got a shiny object, then
(12:45):
that shiny object needs toreflect the real world.
And at least when I was Apple,they had a lot of research going
on into things like this as tolike how to make this thing
interact with the real world
PJ (12:59):
the scene understanding problem remains.
Rob (13:02):
understanding remains, it'll always remain.
It'll just get better over time.
It's like AR without sceneunderstanding is useless.
But has Apple done enough tomake this a consumer friendly
device?
And I think they have.
A few tricks up the sleeve.
They have the whole integrationwith the ecosystem.
They have, integration withvarious immersive movie players.
(13:26):
'cause now they're gonna takefull advantage of this 3D movie
capability.
I do think one of the best usecases of this device is movies,
watching movies and maybe on aplane.
'cause it's like completelyimmersive.
You get 180 degree.
Field of movie experience, youcould kind of move your head
around a little bit and it'llmove the image to give you that
(13:48):
immersive that you're kind ofthere.
They can also do the full 3Dmovies if they want to, but, the
use case for that is difficult.
Think Apple will restrict thetechnology to highlight their
use cases, even if it's not thepure technical use of the
technology.
PJ (14:04):
So what's fascinating about that use case you just mentioned
is that you don't need mixedreality to do the 3D movie case.
Like you could have a VR headsetto do that just as
Rob (14:18):
Yeah, absolutely.
Like it's, it's a VR mixedreality device.
It's the mixed reality justcomes from projecting the real
world into the VR space.
Uh, that is mixed reality andrendering on top of it.
If you want to, the movie usecase could be done by Oculus and
all that, but did Facebook andOculus prior to Facebook, didn't
have the connections that Applehave and didn't have the
(14:39):
ecosystem to integrate into.
So I'm gonna be very interestedto see what the ecosystem is of.
How does this thing integrate.
Do any of these make acompelling case for a must have
application?
The killer app as we call it?
I don't think it does.
I don't think this device wouldbe a hit.
(14:59):
I didn't work on it'cause it wasgonna be a hit.
I worked on it'cause it was cooltechnology and I still don't
think AR is ready for theconsumer space, no matter how
you spin it.
I do think they'll have betterapps than everyone else has had.
It'll be a step in the rightdirection, but it's not gonna be
a.
Like, whoa, look at this.
It's, it's, Apple's gonna do itall better than everybody else.
(15:22):
The betterness will comeliterally from the ecosystem and
the integration with thatecosystem.
Not because they did anythinggroundbreaking, different than
anyone else has already tried.
PJ (15:30):
I sort of stick on one particular point,'cause we
talked a lot earlier about thedifferent focal planes as being,
you know, one difference betweenwhat VR was doing is just
splatting it onto a screenversus what we discussed with
Magic Leap of attempting to havesort of multiple layers of
lenses that describe effectivelymultiple focal lengths.
(15:52):
The Vision Pro does not solve.
It's still one focal plane thatwe're talking about that you're
splatting the image onto.
Rob (16:00):
Yeah, it's very difficult to solve the cure.
Focal plane problem in a VR orXR space.
'cause ultimately there is ascreen in front of your face and
you can do lens in, in front ofthat screen to project it at a
different depth.
The problem is, is your eyes arestill focused on a plane in
front of you.
You take the real world thathas.
(16:20):
All the variable focus andeverything you sp splitt it onto
two camera images, which makesit flat.
Then you put those flat imageson, two flat monitors and you
visualize it as a, a fully 3Dscene and your eyes know that's
not true.
'cause they're focused on a flatplate.
And if you're moving your eyesaround the scene, your eyes are
(16:40):
not changing focus based ondistance as they would in the
real world.
They're focused purely on.
The screen that's in front ofyou, even if the distance
information that's coming fromthat image is changing and it's
like looking at the real worldthrough a dirty window, your
eyes might focus on the glassand the real world's just back
there.
But in that case, it'll all beblurry.
(17:02):
It's'cause your eyes focus of,uh, close up and that will
unfocus the distance.
These are weirdly, infinitelyfocused at a, a very near
distance, and it, it can causeeye strain and again, it gives
me a headache pretty damnquickly.
PJ (17:16):
You mentioned it earlier in terms of the resolution
required, but I do think it'sworthwhile to talk about some of
the numbers.
You talked about it as like amultiple, but in terms of like
what would be required toactually, uh, have what you call
the ultimate screen resolutiongiven.
You know the limitations wehave.
(17:37):
I would love for you to kind oflike reveal exactly what you
calculated out, how many pixelsyou'd need to actually make it
look, you know, real world I.
Rob (17:47):
So pixels become a weird number at some, uh, in this
space because it depends on thefield of view.
If you have a 1920 by 10 80, 1080 p resolution screen, and you
have, let's say 180 degreesfield.
View effectively, like that'sonly about 10 or 11 pixels per
(18:10):
degree of field of view, justthere's nowhere near enough.
They always say, and I thinkthis number is also completely
wrong, you need, effectively 300pixels per inch of one foot, and
that is.
Not even close.
Ideally, you need probably abouta pixel per arc minute is what
(18:33):
they say.
That isn't really correct, buteven that gives you a horizontal
resolution of about 10,000pixels.
So 4K is realistically nowherenear.
The resolution we need.
Now, of course, you can reducethe field of view.
If you make it 90 degrees, thatcould half the size of the, of
(18:55):
the screen, but now you'relimited to this 90 degree field
of view and not the usualperipheral 180 degree that we
have in natural vision.
Technically more than 180, but180 would be a, a great step in
the right direction.
But yeah, for 180 degree fieldof view, like a full hemisphere
in front of you.
You're looking at over 10,000pixels in each resolution, which
(19:18):
would be a hundred millionpixels in the entire display.
So 4K, eight K, yeah, whatever.
Doesn't matter.
We are nowhere near.
So the screen door effect willexist in some capacity at some
distance with some content.
It has to.
There's no way around it.
PJ (19:35):
So for the foreseeable future, AR will remain
relatively near field.
And you know what?
And that might be within, Idon't know, just a few meters
really.
Right.
Rob (19:47):
I mean within a room, I think is the vision Pros use
case, and I think it'll be nearfield.
Like you say, if you do avirtual avatar for chat, it'll
be someone standing at the samedesk as sit at the same desk or
standing next to you orsomething along those lines.
It's gonna be very close, butthe question remains, what if
you take this thing outside?
People will, I guarantee you, Iguarantee you someone will
(20:08):
driving it.
They shouldn't, but I guaranteethey will.
And it's all the killer usecases for AR outdoors.
I want a navigation app thatwill draw on the road and be
like, turn here and literallydraw in the turn lane where I
need to go.
And I literally follow theyellow brick road.
Um, that needs to work withocclusion.
(20:28):
It needs to know about movingobject, high speed, moving
objects.
Um, you've got all the weatherto deal with and.
All the different outdoor thingsthat we encompass, which we
don't see indoors.
There aren't many killer AR usecases for indoors other than
what hopefully Apple will tackleas media playback.
But I think you have a lot tosay on this is movies are
(20:49):
supposed to be social things.
I don't want to sit watch amovie with a headset on my head
with my girlfriend next to mewith a headset on her head.
Watching potentially somethingdifferent.
I have no idea she's evenwatching the same thing.
And it's the same problems 3DTV's had, but much worse.
Like 3D TVs had their, theirthing in the two thousands where
(21:14):
your friends would come over,you'd all wear a pair of
glasses, and you'd watch theseawful movers in 3D.
And if you didn't have a pair ofglasses, the movie wasn't even
watchable.
It wasn't bearable to look atthe screen.
So it made it very antisocialfor people who weren't.
This is a step in the wrongdirection because now not only
is it anti-social, you've gotyour head covered and everybody
(21:35):
needs one of these things.
Again, you might or might not bein the same environment doing
the same thing.
It just seems like a, adystopian use of technology.
I get the use case of you're onan airplane and I can just watch
a movie fully immersed in it andgreat.
That's one very specific usecase for a$3,500 device.
I don't see me choosing it over,laying on the couch, watching
(21:59):
the TV with the speakers blaringand the fireplace on, and my dog
and all the things that I wantnear me when I watch a movie.
And we've talked all throughthis podcast.
We've only been talking aboutgraphics.
We haven't even touched onaudio.
And audio is a huge part of theimmersive experience, and I
(22:19):
believe the Vision Pro is gonnaask you to wear EarPods it.
It has speakers, but.
They can't be very good'causethey're not in your ears.
It's a headset, so it has kindof bone UCT speakers.
You can hear, you can definitelyfigure out what's going on, but
truly immersive audio is gonnarequire more than what it has.
It's gonna require a set ofAirPods or AirPods Pro or
(22:41):
something on your head.
I doubt you can do the pros.
Maybe you can.
It's gonna be a lot of strapsand a lot of things on your
head.
Uh, but, uh, the maxis, sorry,not the pros.
AirPod Pro, what you're gonnahave to wear to get decent
audio, and even that's not greataudio.
I'd much rather have my HiFi,big speakers play an audio in
the room that everyoneexperiences.
(23:02):
I still don't think, no matterhow good you make the audio,
you, unless you have really highquality headphones, then.
You're gonna get the experienceof a, a true Hi-Fi system, and
headphones are going down thatpath of being antisocial again.
PJ (23:21):
This dovetails really nicely into kind of like the, your
question that you've keptbringing up, which is like,
where is the killer app andreally where is the killer app
for the technology where it's attoday?
Because as you point out.
So much of the killer apps wouldbe, far field, uh, ar outside,
you know, navigation and ifwe're containing it.
(23:46):
And a lot of the, the demo isreally like, focus on this if
we're containing this into aspace that's primarily
entertainment.
I started doing a lot ofthinking around the history of
media, or the history ofentertainment really.
Where you can go back as far as,even a prema, technological mass
(24:06):
media days of, you know, your,your mass media may have been
plays or newspapers where youhad effectively this communals,
you know, whether you're all ina, a theater together or you're
watching something and there'sthis kind of unifying effect and
then.
(24:27):
When you have technology, so youhave radio at that point in time
where it's not even confined toa given locale because you can
have stations carrying it acrossa nation.
So now you have, again, acommunal experience that's being
experienced even with folks thatare not right next to you.
Then you have television.
(24:48):
You have movies, you have these,Places where you're gathering
together, either outside yourhome or inside your home to
experience some media together.
And I think we've, I mean,what's really fascinating when I
think about what's happened withsocial networks, because I think
that probably had aspects ofmass communication that were
(25:11):
similar, but over time they havebecome very hyper-personalized.
So we're, we're, we've been downthis road.
Where it's like no one's seeingthe same thing.
We're all getting, you know,different entertainment,
different news, differentpersonalized feeds, and you
could look at like the emergenceof what were happening with,
(25:32):
what's happening with the VisionPro, where it's a device that is
really isolating to a givenperson.
And the same could be said of,of any other VR XR headset.
Because it really is just likeyou alone experiencing that
unless you decide, Hey, I wannabuy, you know, a$3,500 piece of
(25:57):
equipment for everyone in myfamily, which is a pretty cost
prohibitive thing.
I feel like this kind ofisolation, it's gonna be an
amazing tech toy for a lot offolks, I think.
And maybe there's a professionalclass that is gonna get to use
this and it'd be.
Really useful in terms of theirwork.
(26:17):
Maybe they connected to akeyboard or something like that.
And then you have, you know,giant sized monitors.
I don't see the intersectionright now with it being a mask
market consumer device, and I, Isee that as being a big barrier
to its consumer success.
Rob (26:39):
I completely agree.
I think for$3,500, it's out ofthe price point that pretty much
anybody's going to be.
Well, well, let's just see whatit's like.
If it's 500 bucks, a thousanddollars.
I think Oculus is price points.
It is in the realm where techenthusiast be like, yeah, I'll
drop$500 on that just to see andif I never use it, whatever.
But, uh,$3,500, it's out of mostpeople's.
(27:03):
play money fund to uh, to justgo and see.
I mean, maybe the Apple storewill have a bunch of them.
You can go try'em on there.
Again, without that killer app,I don't need an A 3D avatar to
chat.
I can just chat on a keyboard.
But the integration's nice.
It's nice to see it happening.
It's step in the rightdirection, but it isn't, I
think, a step that I'm veryinterested.
PJ (27:25):
Do you have any sense for whether this is, I mean, uh, so
again, I have a family of six.
Uh, if I wanted to buy a VisionPro for everybody, that would be
$20,000.
If I wanted to buy an Oculustoday, I think it's 3000.
So for less than One Vision Pro,I could equip everybody with a
(27:46):
Meta Quest three.
Rob (27:48):
Or you could just have a TV on the wall and you all watch TV
together
PJ (27:51):
or I have a TV that probably costs less than two grand.
do you have any thoughts, like,is this a loss leader for Apple?
Like this is a, you know,high-end concept car, like
Mercedes-Benz might put out withthe idea that this technology
will get then.
Filter down to lower end units.
That makes it more mass market.
(28:12):
It.
It's not necessarily Apple'sstrategy, but I, I do wanna try
and explore whether there'ssomething we're missing here.
Rob (28:18):
So when I was at Apple, I asked some hard questions and
never got any suitable answer,and some of them relate to this.
The first obvious question,which many people have asked is,
if this is a headset device witha puck that's in your pocket,
even if it's wired, why is thatpuck not my phone?
Why is that?
Puck another piece of devicethat has effectively a cell type
(28:43):
cell phone type processor andcell phone type architecture
running an OSS that's derivedfrom the same OSS as the cell
phone.
Why isn't it that?
And the answer is the 14 camerastreams of like there is custom
silicon in there.
So is there a low end versionthat's just a display that's
coming out that drives yourphone, that has less cameras and
things like that?
(29:04):
Then, I don't know, it wasn'ttalked about.
The other thing that, isdifficult in xr, in fact all of
them, vr, a, ar, and, uh, XR isinput, my head's covered.
I can't type, so what sort ofinput device do I use?
Is it accurate enough to see myfingers so I can type on a
(29:24):
virtual keyboard?
Things like that.
I think you avoid those usecases by going down the media
path.
And is there a lower end one?
I really don't know.
It's called the Pro Vision Pro.
There could be a vision.
Looks like there's a MacBook anda MacBook Pro.
Uh, I, I don't know what theirfuture plans are.
(29:46):
It was, this product was stillin development when I left.
So I don't really know.
And if I did say if I did know Isaid it, apple will probably
take this podcast down before wecould even get it up.
PJ (29:55):
Yeah, yeah, yeah, yeah.
I'm not, uh, no, no need tolike, break any NDAs or anything
like that.
This is, this is speculation.
Part of the question comes fromI remember listening to John
Carmack chatting with LexFriedman and discussing his time
at Oculus.
And there was also news articlesthat came out.
(30:17):
So this is not secret that therewas conflicts between Carmack
and Zuckerberg in terms of theparticular realization of VR
with CarMax saying we shouldactually have much lower cost
units.
So what's fascinating to me isthat, compared to a Vision Pro,
(30:37):
you know the Medi Quest three at$600,$500.
It looks vastly cheaper, butit's fascinating to me that
Carmack actually wanted to haveit even cheaper than that to
have a mass market device.
Rob (30:51):
I agree with'em
PJ (30:52):
yeah, I I I, think this is an interesting, like, bullet
point though.
Rob (30:56):
I think if you pick the technology very carefully, good
screens, low latency processing.
Fed may be from a PC instead offrom.
Another computer that's lesshardware to buy already have a
pc, then you can make very highquality, fairly low cost VR
devices.
With modern technology, it'swhen you start putting in custom
(31:19):
processors and standaloneprocessors, the cost inevitably
goes up.
You've got the hardware, you'vegot the development costs.
You have everything in therethat you're feed in, audio,
you're feeding 14 camerascustom, five nanometer silicon,
blah, blah, blah.
The cost has to be high.
How much of it is actuallynecessary for the killer app?
The killer app could be donewithout most of that.
(31:42):
And obviously the, the low costVR device that CarMax talking
about doesn't have xr, it has nocameras.
Uh, so is the killer app justvr?
Do we need the cameras?
Again, I don't know the answerto any of these questions.
I'm more of a technologist onthe AR side, VR side, more than
a use case person, and.
(32:02):
Made plenty of tech that'suseless, but really cool tech.
And this unfortunately might fitin the same category in a much
bigger scale.
Going back to some things youtalked about earlier, the social
experience for, ar vr, like youtalked about, I think Sony did a
good job here with PlayStationVR because they're device as in
HDMI out so you can feed the TVwith what you see in it.
(32:24):
I think it feeds the left eye tothe TV screen.
So all the people in the roomcould help.
They can be like, turn left, dothis, blah, blah, blah.
Makes it a more socialexperience.
Only one of you is getting thefull experience, but it's a step
in the right direction.
It opens up the living room tomore people being involved in
the experience.
So you could switch the headsetaround and all play and all
(32:47):
switch roles.
I think that's a very goodthing.
Uh, I think vr, uh, carmack and.
Oculus do some of this too,because the PC monitor can have
a copy of what's on the
PJ (32:57):
Yeah, you actually can kick it out.
You can cast it to a Chrome cast
Rob (33:01):
Yeah.
So you could do all thesethings.
I don't know if the Vision Procan do that.
I assume it can.
I assume it to be able to takethe same thing, take one eye and
then airplay a two, anotherApple device.
So it's more social.
Hopefully.
Yeah.
Hopefully that's the case.
It opens up the same way thatthe PlayStation originally did
(33:23):
with its HDMI connection.
So again, I don't know if that'sreally supported or will be
supported or down the long roadof things that, uh, they intend
to to implement.
PJ (33:36):
It is interesting that the media examples that I recall
seeing from the keynote reallywere focused around.
Playing movies, playing media, alittle bit of desktop stuff and
the, or the tabletop stuff, Ishould say.
The tabletop ar.
One of the areas that I felt wasunderrepresented was games.
(33:58):
And I feel like it is, slantingmore onto the VR side than the
AR side.
It is one of those like naturalapplications that one could
imagine where.
Maybe your phone becomes yourcontroller and you are, you
know, you're seeing everythingthrough that headset.
There's a part of me
Rob (34:14):
There's lots of use cases and I'm, I've obviously, AR Kit
will be fully supported in itsnew form with the, uh, inside
view of AR for the, uh,StereoVision headsets.
Point of view.
So developers will be able tomake games and will be able to
make their own content.
But I think they took away someof that from the keynote because
(34:38):
of the scene understandingproblem.
These problems are not solved.
They're not solved by anybodyanywhere, although you can
technically solve some of themaltogether.
Not a solved problem.
So I think you'll see AR contentcreeping in.
It'll be in advertising.
Apple do their little ARdisplays for the keynotes and
things like that.
Obviously that will all be thereand, but it's all tech demo.
(35:01):
It's like, yeah, it's like onceyou've seen it, you've seen it
like, okay, don't need to seethat again.
I do think in the future, likeSS AI will play a much bigger
role in AR because it has to,and I think games, games, are
hard because of the whole.
Mobility thing of like if you'redancing around with a headset on
your head, then you've got thewhole wall of a fall over
(35:22):
problem and the motion problemif, I mean, it is a mixed
related device, so effectivelyyou could walk around your house
and solve puzzles and thingslike that.
Do you want to,
PJ (35:34):
Yeah.
Is that actually
Rob (35:35):
There will be use cases for this and people will experiment
with it.
I assume new use cases andthings that we thought would not
be fun end up being fun and viceversa.
So I think developers will runaway and do cool stuff.
I hope Apple give them theability to do so.
Apple's problem with developers,and this may be why there's no
(35:58):
games for the Apple ecosystem asa whole, at least AAA games, is
Apple always assume they are thesmartest person in the room.
They give you SDKs.
Do exactly what they thought youwant to do.
They don't give you tinybuilding blocks where you go, go
and break this.
Go and see what the limits are.
It's all very carefullyconstructed kits where if they
(36:19):
haven't thought of it, you can'tdo it.
And that's one of Apple'sbiggest things.
And they are absolutely not thesmartest people in the, and some
of the big game developers arelike, I don't wanna play with
that.
I also think this came out inthe news this last week, is they
don't really have the mediasystem pinned down either,
because there won't be a Netflixapp, um, Hulu app and
PJ (36:36):
A Spotify app either.
Rob (36:38):
or Spotify app.
Well, Spotify is kind of thatback on the audio pathway.
I don't think Vision Pro hasgreat audio in its natural form.
You have to go buy all the stuffon top of$3,500.
It's gonna be a big problem.
And I don't think it comes witha pair of AirPods.
So you are gonna buy more stuffif you want the full experience.
(36:58):
And for Spotify it's purelyaudio.
I, uh, could see why theywouldn't wanna do it.
And 3D audio is just as old as3D graphics.
PJ (37:05):
This is an article actually that just popped up.
So Disney, believe it or not, isworking on the VR treadmill that
supports virtual users.
From what little I've been ableto tell about it, uh.
I think it's almost like itlooks to me like beads on the
ground that you're like walkingover.
So that'll be really interestingto see if that ever makes it to
(37:28):
a home environment.
Rob (37:31):
See in an urban environment, you could do that.
You could have like, oh, I coulddo a Pacman game, and I'll just,
I'll Pacman, I'll just runaround and pick the dots up or
something.
Now take that outdoors, makethat into, I'm now skiing down a
hill and there are rings that Ican ski through and it's gonna
teach me to do various tricksand turns.
It's all went good until you skioff the edge of the run and go
(37:51):
off a cliff and die'cause the ARwas not quite correct and those
things happened because of thedepth problem.
Uh, I'm a skydiver.
It would be great to have likeinformation in my headset so I
could see what's going on and Icould do it with a head up
display.
It'd be great to point, like,okay, the drop zones behind you
type thing.
It takes the fun out of it.
(38:12):
But there's many use cases orthe quasi indoor outdoor ones I
want to watch a video of fixinga car.
I have some vintage cars.
I like working on them.
I want a video right now.
I watch a YouTube video.
I go and do it.
I watch a YouTube video.
I go and do it.
How about I can watch a YouTubevideo while I'm doing it and it
(38:34):
marks up various parts on theengine, like replace this part,
we replace that part.
And again, scene understanding,am I even looking at the car
when I'm looking at this video?
It's like, should it even playthe markups if it's not sure I'm
looking at the correct thing.
So I think there is newtechnology to be invented and I
(38:54):
think getting these headsets outenables that to be invented.
It could be interesting to seeif that could be standardized
into things that multiple thingscan play.
But as we know, apple doesn'tplay nice with others.
So even if they make technologylike that, it would only be on
their platform.
PJ (39:10):
One of the fascinating things I've found about looking
at Magic Leap, looking atHoloLens, looking at Apple at
this point in time is.
They very much seem like verycool pieces of tech that are in
search of a problem to solve.
And you know, where we've beenable to say, oh, okay, this is
like even through my phone, liketabletop ar.
(39:32):
It's like, okay, cool.
But I'm not using that on adaily basis and it's not
something that's likeparticularly compelling for me
on a daily basis.
So I, I do think that there islike this rich garden that has
been created of visiontechnologies, of graphics
technologies, of pipelines, oftracking.
I think it's awesome.
(39:53):
I just don't quite see how it,it intersects with my daily life
yet though.
And that's been I think one ofthe big, tricky problems with
all of this.
Rob (40:03):
I've said that from day one.
Exactly that.
Like I said, I'm a technologist.
I work on this stuff'cause it'sfun to work on.
There are some cool problems tosolve.
The use case is not my problemand I still haven't seen a
compelling one.
Even knowing how the technologyworks, it could hint, like, push
it this direction'cause.
Fits the technology.
It's still an awful use case,and it's, it is what it is.
(40:27):
It it, it's coming out and we'llsee what happens.
PJ (40:30):
Well, not to put too much of a pun on it, but I think it's a
thing we'll be tracking overtime.
To see how well this thingactually like evolves as the
products get out there.
And who knows, maybe by end ofyear there's something magical
and amazing and it's like, ohyeah, everyone needs a, a Vision
pro or an XR headset and we'llsee what happens.
Rob (40:52):
Yep, I agree.
And before we close, I just wantto go back and address some
numbers.
I just did some back of theenvelope.
Calculations while we weretalking and I was off by an
order of magnitude on therequired display resolution.
PJ (41:05):
Okay, so we said 10,000 pixels, which we're still far
from right now.
What is the actual number youthink, Rob?
Rob (41:13):
there's not, again, it depends on field of view, but if
you
PJ (41:17):
180 180 Field of View.
Rob (41:19):
if you want a 180 field of view and you don't want to have
any of the, higher orderperception artifacts that.
That obviously occur when youhave lower resolution screens,
like motion artifacts andartifacts over time and such
like that you potentially needmaybe a hundred thousand by a
(41:43):
hundred thousand pixels.
So I was literally off by anorder of magnitude
PJ (41:47):
each direction,
Rob (41:49):
in each direction, in each eye too.
So, uh, I don't think I'm gonnabe seeing a.
A hundred by a hundred K displayin my lifetime
PJ (42:00):
uh, especially at the size it would need to be at, in order
to be on one of those headsetsso there's some technical
challenges still to come.
And hopefully we can derive someutility in the meantime.
Rob (42:11):
yep.
And one more thing also I wannapoint out is you mentioned the
tabletop AR from your phone.
That is mixed reality.
That is not ar.
That is true mixed reality.
It is no different on your phonethan it is in your headset,
except you are doing it withdifferent tracking.
You're doing it stereo andthings like that.
It's the same problem, the samethings, and I think you'll get
(42:32):
the same contrived use caseswhen you try to do it in a
headset.
PJ (42:36):
Yeah.
Great, great point.
You know, you mentioned earlier,why not just use the phone?
And I'm curious, like, do youthink there's actually an
opportunity for, let's say theiPhone 15 and above is now, you
know, high speed USBC, which Ithink supports 10 gigabits per
(42:56):
second,
Rob (42:57):
Not even close to enough.
PJ (42:59):
not, not a close to enough.
Not for VR
Rob (43:02):
14.
There are use cases for it.
Yes.
I mean, going back to CarMaxpoint, you could make.
With the available technology,you could use your phone as a
base station and have acertainly VR experience.
PJ (43:15):
Yes.
Yeah.
Rob (43:16):
You could potentially do an AR experience with lower
resolution cameras and fewer ofthem.
Again, what is the use case?
Do we need what it has?
I think carmack hit the nailsquarely on the head where a, an
apple headset that just fed fromyour phone.
Would sell a lot more than a$3,500 headset that has no use
(43:38):
case.
Right now,
PJ (43:39):
Well, what I'm thinking very specifically about is the media
use case where if we're reallytalking about, I wanna watch a
movie on a plane.
I don't need any cameras forthat.
Fundamentally, I don't needcameras looking at my eyes.
I don't need cameras.
Looking at the outside world.
I just need displays, and if Icould just pop that under my
phone and then watch whatever Iwanted, like, okay.
(44:03):
That covers like the media usecase that everyone's talking
about right
Rob (44:07):
That is Carmack's argument right there.
That is it.
You can do the Killing app thatwe have right now for a lot less
and a lot easier than what we'recurrently doing it, dad, we can
do it for it.
Significant cost reduction andget that same movie experience.
All we need for that movieexperience is the display and
(44:29):
potentially the head tracking.
If there's, uh, any 3D componentto it where you can kind of move
your head around or interactwith the scene.
All that's cheap and easilydoable and would be perfectly
doable from an iPhone.
Drive such a display.
Why they don't do that?
I don't know.
I don't think all the extrahardware adds much to the use
case because that is the appthat is the only use case that I
(44:51):
can see the Vision Pro having.
And we can do it much, muchcheaper and I think others will.
It Won't have the Appleecosystem to back it.
And that may be ultimately thekiller feature is the Apple
ecosystem and not any of theselittle Tech demo apps.
PJ (45:08):
I think this is gonna be the big question, is the ecosystem
enough to basically overpowerall of these obstacles that
we've talked about for ar, andit's gonna be an interesting
test case, frankly.
Rob (45:20):
Are you gonna buy one?
PJ (45:21):
I am not as of yet gonna buy one.
Uh, we will see, uh, I willadmit I do wanna play around in
this space.
So I did go the cheaper routeand I just got a Quest three on
order.
Rob (45:33):
Nice.
And I dunno, either, I don't ifwe're gonna buy one, I don't
really fancy drop$3,500 on oneof these things and hopefully
maybe they give one.
I don't know.
We'll, I'll, I'll prod around,see if I can
PJ (45:46):
See if we can get one.
That'd be, I think it'd be funto do like some sort of bake off
between the two making it likethe ecosystem be the
centerpiece.
Like, is, is this enough?
Like, I think that's a reallyvalid thing.
Well, I have a feeling we'regonna be returning to the
subject.
At least once or twice more thisyear as we see the results of
(46:09):
these devices.
Are they catching on?
Are they not?
Rob (46:11):
Yeah, I can't wait to see what people say.
It's gonna be real fun to seehow it pans out and see how pan
my technology gets.
PJ (46:18):
At the end of this, I think there's a two B continued dot,
dot dot.
Rob (46:23):
I think so.
Hi folks.
This is the second of a two-partseries on augmented reality.
If you haven't listened to thefirst, you might want to go back
and listen to that one.
Just to get a little groundwork.
On the other hand, maybe youjust want to jump to the vision
pro and talking about augmentedreality on the apps and the
product side of things.
(00:22):
That's okay too.
But regardless, we wanted togive you a fair warning.
And so now the conclusion.
Of augmented reality.
Ierengaym.
com ierengaym.
(00:43):
com
It's really interesting for you to have connected wanting to
take movie special effects,which as you put it, come across
perfectly.
But you know, one of the hitcheswith that is that all movies are
(01:04):
basically all 2D images, right?
So we do this, this sort ofperfect combination, but it
ultimately slams down to a 2Dimage.
And I felt that was aninteresting connection point to
take us right back to mixedreality at this point in time.
So we've done vr, we've done ar.
(01:25):
Now let's talk about the magicthat is the Vision Pro or what
it's actually doing.
Rob (01:31):
The Vision Pro, it was announced for developers last
Friday, the 19th of January, Ibelieve it was.
And it'll be for sale on, or atleast for order on the 3rd of
February.
And then ultimately people willget these in the hands and we'll
start to get reviews of them andall the issues potentially exist
(01:56):
start to come up.
It's gonna be very interestingto see how Apple, with their
fully self-contained ecosystem.
And how is this gonna interactwith everything else?
And I've seen a lot of this, Ihaven't seen it in a couple
years, but, it's definitelygonna integrate with the
ecosystem.
So.
(02:16):
That brings us to the questionof what is the vision pro?
Where did it come from?
And there's always been rumorsabout Apple making glasses, and
they're just gonna be a pair ofsuper thin framed, ultra
stylistic glasses that you canwear.
And as we've talked about, it'snot actually possible even for
Apple because there's so muchhardware that has to be in the
(02:39):
head space and it will betethered.
You did make a thin pair ofglasses'cause it would need to
be powered.
There'd be no room for batteriesor things like that but the
Vision Pro isn't a pair ofglasses as we've all seen.
It's, it's a headset.
It's technically a VR headset ithas no visible light from the
outside world into the actualdevice and in ultimately into
(03:02):
your eyes.
In fact, it has a very goodlight seal around the device to
stop outside light getting in.
So why are we calling it a mixedreality device when all in
intent and purpose, it's, it's aVR device and the reason is
cameras to be specific.
14 cameras on that device.
(03:23):
That's a awful amount ofbandwidth to handle all the
time.
That's why it has customsilicon.
Those cameras are dedicated toall different jobs.
Some of them face out, some ofthem face down to look at your
hands, some face looking at youreyes.
Uh, some are infrared.
There's a depth camera.
(03:43):
There's many forms of cameraslooking all directions,
different frameworks, differentresolutions.
So.
What they're calling mixedreality or XR or whatever other
name they want to give.
It is effectively going back towhat I said about video and
video playback on the Oculus ina standard VR device in real
(04:03):
time.
So we're taking live feeds fromcameras, projecting it onto the
displays, and so it looks likeyou're not wearing a headset.
You could see the real worldand.
It's more than that becausethat's just pass through camera.
Oculus played with this.
There's various things that theyplayed with years ago that
(04:25):
people 3D printed things and putcameras on.
There's a whole bunch ofproblems to solve here with
projection and making sure thatit looks the same as it would
look if you were to.
Not have the headset on.
So you've gotta undo thedistortions on the lenses, on
the cameras and processor canput it back into your eyes.
(04:47):
Now obviously there is latencythere, there is a little bit of
latency in that process.
How fast can you do that?
In reality?
Very fast.
'cause you really taking a framefrom one thing, adding it to a
display and that can be veryquick.
In fact, apple have a patternmode.
If it crashes, it willautomatically, instantly
Hardware.
Pass through so you're not stuckwith this device on your head
(05:10):
that you can't actually see outof.
So it's one thing that no otherdevice has.
Like Oculus, if it crashes, youend up with a crash still image
on your face and you've gottatake the headset off.
And in the process you mightfall over.
Whereas Apple have this watchdogsystem where if you don't
respond to the timer quickenough, it will automatically in
(05:31):
hardware go at this point.
Instantaneous pass through mode.
So it just looks like it's apair of glasses at that point.
'cause you're looking at thereal world, but you're looking
through a pair of cameras,you're not looking at the real
world.
Like I said, there's no way forlight to get in this device.
PJ (05:46):
Earlier, Rob, we talked about 10 seconds as being that,
you know, 10 seconds are underas being kind of that space you
want it to be.
Maybe 16, sorry, not seconds.
16.
10.
16 milliseconds to avoid theseasickness.
So I take it that with the passthrough mode, we're well
underneath that at that point in
Rob (06:05):
Oh yeah.
It's, it's instantaneous.
it's it's scanned one and scanthe other.
It's just like boom, like superlow agency.
And so that's one thing it hasand.
In all intended purpose, it doesall the things a VR device does.
It's very integrated with theApple ecosystem.
Uh, it'll have iMessage support,it plays iPad apps in 2D space
(06:30):
and all that.
It has a head start over allcompeting devices, and it has
Apple's name behind it, but itstill has all the same problems.
For example, if you open a webbrowser, how high resolution
does the display have to bebefore you can read text on a
window that's six feet away fromyou?
And that's two 4K displays veryhigh resolution, but nowhere
(06:54):
near high enough resolution.
To be able to read text at adistance, just put your laptop
screen and walk across the otherside of the room and see if you
can read it.
And that's actually a high riskscreen at a distance.
Now, make that a low risk screeneffectively and you can't read
the text.
And text is incredibly difficultto handle in a VR device.
(07:14):
All of the devices, up untilnow, magic Leap and Oculus all
had this same problem we used tohaving a.
Laptop screen, which ispotentially higher than 4K,
right?
In a phase, it's a foot away.
Now you've got that entireresolution.
Rather than being a tiny littlesolid angle of your actual
vision, it's now the wholething.
(07:36):
And which pro has a very widefield of view.
So if you factor out like howmany pixels per degree, that
display needs to be like fivetimes the resolution before you
can reliably handle text at.
Where you'd expect to be able toread it.
Obviously, if you put a a letterpiece of paper a hundred feet
away, you're not gonna be ableto read it in the real world
(07:57):
either.
So resolution at that scalebecomes less relevant, but five,
10 feet away with a a bigvirtual desktop, you, you're
gonna be wanting to be able toread the text and Apple do a
very good job of it, but it'sstill gonna have some of these
classic artifacts.
That the other VR devices havewhen it comes to small objects
(08:20):
door effect is minimized, butit's still there if you look for
it.
That passthrough mode that wejust talked about is how that,
in the keynote day, said thatsomeone, if you are watching a,
a movie in full immersive mode,someone walked in the room.
It'll blend into the real worldso you can kind of see what's
going on in the real worldwithout taking the headset off.
(08:43):
That same pastoral system is howit's doing that.
It's just blending Canberraimages with virtual images and
giving you that, uh, a a viewingto the real world.
But, but again, that's stillkind of vr.
What's mixed reality?
Well, mixed reality is, well,now we have this.
Synthetic vision.
(09:04):
Basically we have these camerasthat are feeding in.
We can render effectivelyaugmented reality style on top
of those camera images anddisplay that in your face.
And visibly it looks just kindalike ar.
But in a lot of ways it's a loteasier because first of all, you
(09:24):
can manage latency a lot easier.
In true ar.
The real world is the realworld.
It's gonna do what it does.
In mixed reality, you can delaythe frames from the camera up
until the latency limits thatare toleratable for VR
basically.
So if you take a frame from acamera that can be timestamped,
(09:49):
this is exactly the time thatframe was captured.
You can render at exactly thattimestamp, so there's no
shimmying around in the scene.
It's not like the virtualobjects are gonna move with
respect to the captured framebecause you know the exact
timestamps and you can renderall that, process it all, and
then display it 10 milliseconds,20 milliseconds later, and the
(10:12):
frame looks complete.
So the whole virtual objectsskied around due to the real
world not having latency and thevirtual object having latency.
It's fixed.
The whole thing now has latencyin the same way that VR does and
PJ (10:28):
so to, put a fine point on this, we are cheating in the
sense that we're delaying thereal world, you know, 10, 20
milliseconds.
And that's okay because that'sstill within the tolerance of
seasickness, but we get to usethat to synchronize the virtual
and the real feed together.
Rob (10:50):
yeah.
Basically we have a stream offrames coming in from the real
world.
None of them are visible in realtime.
They're all delayed slightly,and that delay allows us to
render on top of them.
And we still have to doeverything we have to do for
real.
Ar We have to render, take twocamera images, render stereo on
top of those.
They get projected in the glass.
And now we, you've got thiscamera based reality with
(11:13):
virtual objects in it.
So the latency is a lot easierto manage.
You still have to do the headtracking, head prediction and
all that just to get the numberdown low enough.
But, uh, the rendering side onthe, just the purely technical
side of this, generating animage is significantly better
because you can have, I knowthis frame needs to have a 3D
(11:37):
image at this timestamp, and Ican blend them perfectly.
You still have all the existingproblems that we talked about
with true ar.
Just because you are in cameraspace instead of real world
light space doesn't mean theocclusion problem goes away,
depth clipping and all thingslike that still needs to occur
PJ (12:00):
The shadow problem's easier though.
Rob (12:03):
the shadow problem is now fixable because.
The image is now virtual.
The camera image is just a pixelbased image.
It's not the real world.
So we can subtract light fromit, we can replace pixels with
black, and if we have a fineenough understanding, then
lighting becomes much easierbecause we can do blended light,
(12:23):
we can do subtractive light.
It's not just additive like itwas in the true AR sense.
So visual effects get better.
Dynamic range gets better'cause.
We're not just adding lighttowards, it's already there and
you still have all the sameproblems of if a light's
flashing in the distance andyou've got a shiny object, then
(12:45):
that shiny object needs toreflect the real world.
And at least when I was Apple,they had a lot of research going
on into things like this as tolike how to make this thing
interact with the real world
PJ (12:59):
the scene understanding problem remains.
Rob (13:02):
understanding remains, it'll always remain.
It'll just get better over time.
It's like AR without sceneunderstanding is useless.
But has Apple done enough tomake this a consumer friendly
device?
And I think they have.
A few tricks up the sleeve.
They have the whole integrationwith the ecosystem.
They have, integration withvarious immersive movie players.
(13:26):
'cause now they're gonna takefull advantage of this 3D movie
capability.
I do think one of the best usecases of this device is movies,
watching movies and maybe on aplane.
'cause it's like completelyimmersive.
You get 180 degree.
Field of movie experience, youcould kind of move your head
around a little bit and it'llmove the image to give you that
(13:48):
immersive that you're kind ofthere.
They can also do the full 3Dmovies if they want to, but, the
use case for that is difficult.
Think Apple will restrict thetechnology to highlight their
use cases, even if it's not thepure technical use of the
technology.
PJ (14:04):
So what's fascinating about that use case you just mentioned
is that you don't need mixedreality to do the 3D movie case.
Like you could have a VR headsetto do that just as
Rob (14:18):
Yeah, absolutely.
Like it's, it's a VR mixedreality device.
It's the mixed reality justcomes from projecting the real
world into the VR space.
Uh, that is mixed reality andrendering on top of it.
If you want to, the movie usecase could be done by Oculus and
all that, but did Facebook andOculus prior to Facebook, didn't
have the connections that Applehave and didn't have the
(14:39):
ecosystem to integrate into.
So I'm gonna be very interestedto see what the ecosystem is of.
How does this thing integrate.
Do any of these make acompelling case for a must have
application?
The killer app as we call it?
I don't think it does.
I don't think this device wouldbe a hit.
(14:59):
I didn't work on it'cause it wasgonna be a hit.
I worked on it'cause it was cooltechnology and I still don't
think AR is ready for theconsumer space, no matter how
you spin it.
I do think they'll have betterapps than everyone else has had.
It'll be a step in the rightdirection, but it's not gonna be
a.
Like, whoa, look at this.
It's, it's, Apple's gonna do itall better than everybody else.
(15:22):
The betterness will comeliterally from the ecosystem and
the integration with thatecosystem.
Not because they did anythinggroundbreaking, different than
anyone else has already tried.
PJ (15:30):
I sort of stick on one particular point,'cause we
talked a lot earlier about thedifferent focal planes as being,
you know, one difference betweenwhat VR was doing is just
splatting it onto a screenversus what we discussed with
Magic Leap of attempting to havesort of multiple layers of
lenses that describe effectivelymultiple focal lengths.
(15:52):
The Vision Pro does not solve.
It's still one focal plane thatwe're talking about that you're
splatting the image onto.
Rob (16:00):
Yeah, it's very difficult to solve the cure.
Focal plane problem in a VR orXR space.
'cause ultimately there is ascreen in front of your face and
you can do lens in, in front ofthat screen to project it at a
different depth.
The problem is, is your eyes arestill focused on a plane in
front of you.
You take the real world thathas.
(16:20):
All the variable focus andeverything you sp splitt it onto
two camera images, which makesit flat.
Then you put those flat imageson, two flat monitors and you
visualize it as a, a fully 3Dscene and your eyes know that's
not true.
'cause they're focused on a flatplate.
And if you're moving your eyesaround the scene, your eyes are
(16:40):
not changing focus based ondistance as they would in the
real world.
They're focused purely on.
The screen that's in front ofyou, even if the distance
information that's coming fromthat image is changing and it's
like looking at the real worldthrough a dirty window, your
eyes might focus on the glassand the real world's just back
there.
But in that case, it'll all beblurry.
(17:02):
It's'cause your eyes focus of,uh, close up and that will
unfocus the distance.
These are weirdly, infinitelyfocused at a, a very near
distance, and it, it can causeeye strain and again, it gives
me a headache pretty damnquickly.
PJ (17:16):
You mentioned it earlier in terms of the resolution
required, but I do think it'sworthwhile to talk about some of
the numbers.
You talked about it as like amultiple, but in terms of like
what would be required toactually, uh, have what you call
the ultimate screen resolutiongiven.
You know the limitations wehave.
(17:37):
I would love for you to kind oflike reveal exactly what you
calculated out, how many pixelsyou'd need to actually make it
look, you know, real world I.
Rob (17:47):
So pixels become a weird number at some, uh, in this
space because it depends on thefield of view.
If you have a 1920 by 10 80, 1080 p resolution screen, and you
have, let's say 180 degreesfield.
View effectively, like that'sonly about 10 or 11 pixels per
(18:10):
degree of field of view, justthere's nowhere near enough.
They always say, and I thinkthis number is also completely
wrong, you need, effectively 300pixels per inch of one foot, and
that is.
Not even close.
Ideally, you need probably abouta pixel per arc minute is what
(18:33):
they say.
That isn't really correct, buteven that gives you a horizontal
resolution of about 10,000pixels.
So 4K is realistically nowherenear.
The resolution we need.
Now, of course, you can reducethe field of view.
If you make it 90 degrees, thatcould half the size of the, of
(18:55):
the screen, but now you'relimited to this 90 degree field
of view and not the usualperipheral 180 degree that we
have in natural vision.
Technically more than 180, but180 would be a, a great step in
the right direction.
But yeah, for 180 degree fieldof view, like a full hemisphere
in front of you.
You're looking at over 10,000pixels in each resolution, which
(19:18):
would be a hundred millionpixels in the entire display.
So 4K, eight K, yeah, whatever.
Doesn't matter.
We are nowhere near.
So the screen door effect willexist in some capacity at some
distance with some content.
It has to.
There's no way around it.
PJ (19:35):
So for the foreseeable future, AR will remain
relatively near field.
And you know what?
And that might be within, Idon't know, just a few meters
really.
Right.
Rob (19:47):
I mean within a room, I think is the vision Pros use
case, and I think it'll be nearfield.
Like you say, if you do avirtual avatar for chat, it'll
be someone standing at the samedesk as sit at the same desk or
standing next to you orsomething along those lines.
It's gonna be very close, butthe question remains, what if
you take this thing outside?
People will, I guarantee you, Iguarantee you someone will
(20:08):
driving it.
They shouldn't, but I guaranteethey will.
And it's all the killer usecases for AR outdoors.
I want a navigation app thatwill draw on the road and be
like, turn here and literallydraw in the turn lane where I
need to go.
And I literally follow theyellow brick road.
Um, that needs to work withocclusion.
(20:28):
It needs to know about movingobject, high speed, moving
objects.
Um, you've got all the weatherto deal with and.
All the different outdoor thingsthat we encompass, which we
don't see indoors.
There aren't many killer AR usecases for indoors other than
what hopefully Apple will tackleas media playback.
But I think you have a lot tosay on this is movies are
(20:49):
supposed to be social things.
I don't want to sit watch amovie with a headset on my head
with my girlfriend next to mewith a headset on her head.
Watching potentially somethingdifferent.
I have no idea she's evenwatching the same thing.
And it's the same problems 3DTV's had, but much worse.
Like 3D TVs had their, theirthing in the two thousands where
(21:14):
your friends would come over,you'd all wear a pair of
glasses, and you'd watch theseawful movers in 3D.
And if you didn't have a pair ofglasses, the movie wasn't even
watchable.
It wasn't bearable to look atthe screen.
So it made it very antisocialfor people who weren't.
This is a step in the wrongdirection because now not only
is it anti-social, you've gotyour head covered and everybody
(21:35):
needs one of these things.
Again, you might or might not bein the same environment doing
the same thing.
It just seems like a, adystopian use of technology.
I get the use case of you're onan airplane and I can just watch
a movie fully immersed in it andgreat.
That's one very specific usecase for a$3,500 device.
I don't see me choosing it over,laying on the couch, watching
(21:59):
the TV with the speakers blaringand the fireplace on, and my dog
and all the things that I wantnear me when I watch a movie.
And we've talked all throughthis podcast.
We've only been talking aboutgraphics.
We haven't even touched onaudio.
And audio is a huge part of theimmersive experience, and I
(22:19):
believe the Vision Pro is gonnaask you to wear EarPods it.
It has speakers, but.
They can't be very good'causethey're not in your ears.
It's a headset, so it has kindof bone UCT speakers.
You can hear, you can definitelyfigure out what's going on, but
truly immersive audio is gonnarequire more than what it has.
It's gonna require a set ofAirPods or AirPods Pro or
(22:41):
something on your head.
I doubt you can do the pros.
Maybe you can.
It's gonna be a lot of strapsand a lot of things on your
head.
Uh, but, uh, the maxis, sorry,not the pros.
AirPod Pro, what you're gonnahave to wear to get decent
audio, and even that's not greataudio.
I'd much rather have my HiFi,big speakers play an audio in
the room that everyoneexperiences.
(23:02):
I still don't think, no matterhow good you make the audio,
you, unless you have really highquality headphones, then.
You're gonna get the experienceof a, a true Hi-Fi system, and
headphones are going down thatpath of being antisocial again.
PJ (23:21):
This dovetails really nicely into kind of like the, your
question that you've keptbringing up, which is like,
where is the killer app andreally where is the killer app
for the technology where it's attoday?
Because as you point out.
So much of the killer apps wouldbe, far field, uh, ar outside,
you know, navigation and ifwe're containing it.
(23:46):
And a lot of the, the demo isreally like, focus on this if
we're containing this into aspace that's primarily
entertainment.
I started doing a lot ofthinking around the history of
media, or the history ofentertainment really.
Where you can go back as far as,even a prema, technological mass
(24:06):
media days of, you know, your,your mass media may have been
plays or newspapers where youhad effectively this communals,
you know, whether you're all ina, a theater together or you're
watching something and there'sthis kind of unifying effect and
then.
(24:27):
When you have technology, so youhave radio at that point in time
where it's not even confined toa given locale because you can
have stations carrying it acrossa nation.
So now you have, again, acommunal experience that's being
experienced even with folks thatare not right next to you.
Then you have television.
(24:48):
You have movies, you have these,Places where you're gathering
together, either outside yourhome or inside your home to
experience some media together.
And I think we've, I mean,what's really fascinating when I
think about what's happened withsocial networks, because I think
that probably had aspects ofmass communication that were
(25:11):
similar, but over time they havebecome very hyper-personalized.
So we're, we're, we've been downthis road.
Where it's like no one's seeingthe same thing.
We're all getting, you know,different entertainment,
different news, differentpersonalized feeds, and you
could look at like the emergenceof what were happening with,
(25:32):
what's happening with the VisionPro, where it's a device that is
really isolating to a givenperson.
And the same could be said of,of any other VR XR headset.
Because it really is just likeyou alone experiencing that
unless you decide, Hey, I wannabuy, you know, a$3,500 piece of
(25:57):
equipment for everyone in myfamily, which is a pretty cost
prohibitive thing.
I feel like this kind ofisolation, it's gonna be an
amazing tech toy for a lot offolks, I think.
And maybe there's a professionalclass that is gonna get to use
this and it'd be.
Really useful in terms of theirwork.
(26:17):
Maybe they connected to akeyboard or something like that.
And then you have, you know,giant sized monitors.
I don't see the intersectionright now with it being a mask
market consumer device, and I, Isee that as being a big barrier
to its consumer success.
Rob (26:39):
I completely agree.
I think for$3,500, it's out ofthe price point that pretty much
anybody's going to be.
Well, well, let's just see whatit's like.
If it's 500 bucks, a thousanddollars.
I think Oculus is price points.
It is in the realm where techenthusiast be like, yeah, I'll
drop$500 on that just to see andif I never use it, whatever.
But, uh,$3,500, it's out of mostpeople's.
(27:03):
play money fund to uh, to justgo and see.
I mean, maybe the Apple storewill have a bunch of them.
You can go try'em on there.
Again, without that killer app,I don't need an A 3D avatar to
chat.
I can just chat on a keyboard.
But the integration's nice.
It's nice to see it happening.
It's step in the rightdirection, but it isn't, I
think, a step that I'm veryinterested.
PJ (27:25):
Do you have any sense for whether this is, I mean, uh, so
again, I have a family of six.
Uh, if I wanted to buy a VisionPro for everybody, that would be
$20,000.
If I wanted to buy an Oculustoday, I think it's 3000.
So for less than One Vision Pro,I could equip everybody with a
(27:46):
Meta Quest three.
Rob (27:48):
Or you could just have a TV on the wall and you all watch TV
together
PJ (27:51):
or I have a TV that probably costs less than two grand.
do you have any thoughts, like,is this a loss leader for Apple?
Like this is a, you know,high-end concept car, like
Mercedes-Benz might put out withthe idea that this technology
will get then.
Filter down to lower end units.
That makes it more mass market.
(28:12):
It.
It's not necessarily Apple'sstrategy, but I, I do wanna try
and explore whether there'ssomething we're missing here.
Rob (28:18):
So when I was at Apple, I asked some hard questions and
never got any suitable answer,and some of them relate to this.
The first obvious question,which many people have asked is,
if this is a headset device witha puck that's in your pocket,
even if it's wired, why is thatpuck not my phone?
Why is that?
Puck another piece of devicethat has effectively a cell type
(28:43):
cell phone type processor andcell phone type architecture
running an OSS that's derivedfrom the same OSS as the cell
phone.
Why isn't it that?
And the answer is the 14 camerastreams of like there is custom
silicon in there.
So is there a low end versionthat's just a display that's
coming out that drives yourphone, that has less cameras and
things like that?
(29:04):
Then, I don't know, it wasn'ttalked about.
The other thing that, isdifficult in xr, in fact all of
them, vr, a, ar, and, uh, XR isinput, my head's covered.
I can't type, so what sort ofinput device do I use?
Is it accurate enough to see myfingers so I can type on a
(29:24):
virtual keyboard?
Things like that.
I think you avoid those usecases by going down the media
path.
And is there a lower end one?
I really don't know.
It's called the Pro Vision Pro.
There could be a vision.
Looks like there's a MacBook anda MacBook Pro.
Uh, I, I don't know what theirfuture plans are.
(29:46):
It was, this product was stillin development when I left.
So I don't really know.
And if I did say if I did know Isaid it, apple will probably
take this podcast down before wecould even get it up.
PJ (29:55):
Yeah, yeah, yeah, yeah.
I'm not, uh, no, no need tolike, break any NDAs or anything
like that.
This is, this is speculation.
Part of the question comes fromI remember listening to John
Carmack chatting with LexFriedman and discussing his time
at Oculus.
And there was also news articlesthat came out.
(30:17):
So this is not secret that therewas conflicts between Carmack
and Zuckerberg in terms of theparticular realization of VR
with CarMax saying we shouldactually have much lower cost
units.
So what's fascinating to me isthat, compared to a Vision Pro,
(30:37):
you know the Medi Quest three at$600,$500.
It looks vastly cheaper, butit's fascinating to me that
Carmack actually wanted to haveit even cheaper than that to
have a mass market device.
Rob (30:51):
I agree with'em
PJ (30:52):
yeah, I I I, think this is an interesting, like, bullet
point though.
Rob (30:56):
I think if you pick the technology very carefully, good
screens, low latency processing.
Fed may be from a PC instead offrom.
Another computer that's lesshardware to buy already have a
pc, then you can make very highquality, fairly low cost VR
devices.
With modern technology, it'swhen you start putting in custom
(31:19):
processors and standaloneprocessors, the cost inevitably
goes up.
You've got the hardware, you'vegot the development costs.
You have everything in therethat you're feed in, audio,
you're feeding 14 camerascustom, five nanometer silicon,
blah, blah, blah.
The cost has to be high.
How much of it is actuallynecessary for the killer app?
The killer app could be donewithout most of that.
(31:42):
And obviously the, the low costVR device that CarMax talking
about doesn't have xr, it has nocameras.
Uh, so is the killer app justvr?
Do we need the cameras?
Again, I don't know the answerto any of these questions.
I'm more of a technologist onthe AR side, VR side, more than
a use case person, and.
(32:02):
Made plenty of tech that'suseless, but really cool tech.
And this unfortunately might fitin the same category in a much
bigger scale.
Going back to some things youtalked about earlier, the social
experience for, ar vr, like youtalked about, I think Sony did a
good job here with PlayStationVR because they're device as in
HDMI out so you can feed the TVwith what you see in it.
(32:24):
I think it feeds the left eye tothe TV screen.
So all the people in the roomcould help.
They can be like, turn left, dothis, blah, blah, blah.
Makes it a more socialexperience.
Only one of you is getting thefull experience, but it's a step
in the right direction.
It opens up the living room tomore people being involved in
the experience.
So you could switch the headsetaround and all play and all
(32:47):
switch roles.
I think that's a very goodthing.
Uh, I think vr, uh, carmack and.
Oculus do some of this too,because the PC monitor can have
a copy of what's on the
PJ (32:57):
Yeah, you actually can kick it out.
You can cast it to a Chrome cast
Rob (33:01):
Yeah.
So you could do all thesethings.
I don't know if the Vision Procan do that.
I assume it can.
I assume it to be able to takethe same thing, take one eye and
then airplay a two, anotherApple device.
So it's more social.
Hopefully.
Yeah.
Hopefully that's the case.
It opens up the same way thatthe PlayStation originally did
(33:23):
with its HDMI connection.
So again, I don't know if that'sreally supported or will be
supported or down the long roadof things that, uh, they intend
to to implement.
PJ (33:36):
It is interesting that the media examples that I recall
seeing from the keynote reallywere focused around.
Playing movies, playing media, alittle bit of desktop stuff and
the, or the tabletop stuff, Ishould say.
The tabletop ar.
One of the areas that I felt wasunderrepresented was games.
(33:58):
And I feel like it is, slantingmore onto the VR side than the
AR side.
It is one of those like naturalapplications that one could
imagine where.
Maybe your phone becomes yourcontroller and you are, you
know, you're seeing everythingthrough that headset.
There's a part of me
Rob (34:14):
There's lots of use cases and I'm, I've obviously, AR Kit
will be fully supported in itsnew form with the, uh, inside
view of AR for the, uh,StereoVision headsets.
Point of view.
So developers will be able tomake games and will be able to
make their own content.
But I think they took away someof that from the keynote because
(34:38):
of the scene understandingproblem.
These problems are not solved.
They're not solved by anybodyanywhere, although you can
technically solve some of themaltogether.
Not a solved problem.
So I think you'll see AR contentcreeping in.
It'll be in advertising.
Apple do their little ARdisplays for the keynotes and
things like that.
Obviously that will all be thereand, but it's all tech demo.
(35:01):
It's like, yeah, it's like onceyou've seen it, you've seen it
like, okay, don't need to seethat again.
I do think in the future, likeSS AI will play a much bigger
role in AR because it has to,and I think games, games, are
hard because of the whole.
Mobility thing of like if you'redancing around with a headset on
your head, then you've got thewhole wall of a fall over
(35:22):
problem and the motion problemif, I mean, it is a mixed
related device, so effectivelyyou could walk around your house
and solve puzzles and thingslike that.
Do you want to,
PJ (35:34):
Yeah.
Is that actually
Rob (35:35):
There will be use cases for this and people will experiment
with it.
I assume new use cases andthings that we thought would not
be fun end up being fun and viceversa.
So I think developers will runaway and do cool stuff.
I hope Apple give them theability to do so.
Apple's problem with developers,and this may be why there's no
(35:58):
games for the Apple ecosystem asa whole, at least AAA games, is
Apple always assume they are thesmartest person in the room.
They give you SDKs.
Do exactly what they thought youwant to do.
They don't give you tinybuilding blocks where you go, go
and break this.
Go and see what the limits are.
It's all very carefullyconstructed kits where if they
(36:19):
haven't thought of it, you can'tdo it.
And that's one of Apple'sbiggest things.
And they are absolutely not thesmartest people in the, and some
of the big game developers arelike, I don't wanna play with
that.
I also think this came out inthe news this last week, is they
don't really have the mediasystem pinned down either,
because there won't be a Netflixapp, um, Hulu app and
PJ (36:36):
A Spotify app either.
Rob (36:38):
or Spotify app.
Well, Spotify is kind of thatback on the audio pathway.
I don't think Vision Pro hasgreat audio in its natural form.
You have to go buy all the stuffon top of$3,500.
It's gonna be a big problem.
And I don't think it comes witha pair of AirPods.
So you are gonna buy more stuffif you want the full experience.
(36:58):
And for Spotify it's purelyaudio.
I, uh, could see why theywouldn't wanna do it.
And 3D audio is just as old as3D graphics.
PJ (37:05):
This is an article actually that just popped up.
So Disney, believe it or not, isworking on the VR treadmill that
supports virtual users.
From what little I've been ableto tell about it, uh.
I think it's almost like itlooks to me like beads on the
ground that you're like walkingover.
So that'll be really interestingto see if that ever makes it to
(37:28):
a home environment.
Rob (37:31):
See in an urban environment, you could do that.
You could have like, oh, I coulddo a Pacman game, and I'll just,
I'll Pacman, I'll just runaround and pick the dots up or
something.
Now take that outdoors, makethat into, I'm now skiing down a
hill and there are rings that Ican ski through and it's gonna
teach me to do various tricksand turns.
It's all went good until you skioff the edge of the run and go
(37:51):
off a cliff and die'cause the ARwas not quite correct and those
things happened because of thedepth problem.
Uh, I'm a skydiver.
It would be great to have likeinformation in my headset so I
could see what's going on and Icould do it with a head up
display.
It'd be great to point, like,okay, the drop zones behind you
type thing.
It takes the fun out of it.
(38:12):
But there's many use cases orthe quasi indoor outdoor ones I
want to watch a video of fixinga car.
I have some vintage cars.
I like working on them.
I want a video right now.
I watch a YouTube video.
I go and do it.
I watch a YouTube video.
I go and do it.
How about I can watch a YouTubevideo while I'm doing it and it
(38:34):
marks up various parts on theengine, like replace this part,
we replace that part.
And again, scene understanding,am I even looking at the car
when I'm looking at this video?
It's like, should it even playthe markups if it's not sure I'm
looking at the correct thing.
So I think there is newtechnology to be invented and I
(38:54):
think getting these headsets outenables that to be invented.
It could be interesting to seeif that could be standardized
into things that multiple thingscan play.
But as we know, apple doesn'tplay nice with others.
So even if they make technologylike that, it would only be on
their platform.
PJ (39:10):
One of the fascinating things I've found about looking
at Magic Leap, looking atHoloLens, looking at Apple at
this point in time is.
They very much seem like verycool pieces of tech that are in
search of a problem to solve.
And you know, where we've beenable to say, oh, okay, this is
like even through my phone, liketabletop ar.
(39:32):
It's like, okay, cool.
But I'm not using that on adaily basis and it's not
something that's likeparticularly compelling for me
on a daily basis.
So I, I do think that there islike this rich garden that has
been created of visiontechnologies, of graphics
technologies, of pipelines, oftracking.
I think it's awesome.
(39:53):
I just don't quite see how it,it intersects with my daily life
yet though.
And that's been I think one ofthe big, tricky problems with
all of this.
Rob (40:03):
I've said that from day one.
Exactly that.
Like I said, I'm a technologist.
I work on this stuff'cause it'sfun to work on.
There are some cool problems tosolve.
The use case is not my problemand I still haven't seen a
compelling one.
Even knowing how the technologyworks, it could hint, like, push
it this direction'cause.
Fits the technology.
It's still an awful use case,and it's, it is what it is.
(40:27):
It it, it's coming out and we'llsee what happens.
PJ (40:30):
Well, not to put too much of a pun on it, but I think it's a
thing we'll be tracking overtime.
To see how well this thingactually like evolves as the
products get out there.
And who knows, maybe by end ofyear there's something magical
and amazing and it's like, ohyeah, everyone needs a, a Vision
pro or an XR headset and we'llsee what happens.
Rob (40:52):
Yep, I agree.
And before we close, I just wantto go back and address some
numbers.
I just did some back of theenvelope.
Calculations while we weretalking and I was off by an
order of magnitude on therequired display resolution.
PJ (41:05):
Okay, so we said 10,000 pixels, which we're still far
from right now.
What is the actual number youthink, Rob?
Rob (41:13):
there's not, again, it depends on field of view, but if
you
PJ (41:17):
180 180 Field of View.
Rob (41:19):
if you want a 180 field of view and you don't want to have
any of the, higher orderperception artifacts that.
That obviously occur when youhave lower resolution screens,
like motion artifacts andartifacts over time and such
like that you potentially needmaybe a hundred thousand by a
(41:43):
hundred thousand pixels.
So I was literally off by anorder of magnitude
PJ (41:47):
each direction,
Rob (41:49):
in each direction, in each eye too.
So, uh, I don't think I'm gonnabe seeing a.
A hundred by a hundred K displayin my lifetime
PJ (42:00):
uh, especially at the size it would need to be at, in order
to be on one of those headsetsso there's some technical
challenges still to come.
And hopefully we can derive someutility in the meantime.
Rob (42:11):
yep.
And one more thing also I wannapoint out is you mentioned the
tabletop AR from your phone.
That is mixed reality.
That is not ar.
That is true mixed reality.
It is no different on your phonethan it is in your headset,
except you are doing it withdifferent tracking.
You're doing it stereo andthings like that.
It's the same problem, the samethings, and I think you'll get
(42:32):
the same contrived use caseswhen you try to do it in a
headset.
PJ (42:36):
Yeah.
Great, great point.
You know, you mentioned earlier,why not just use the phone?
And I'm curious, like, do youthink there's actually an
opportunity for, let's say theiPhone 15 and above is now, you
know, high speed USBC, which Ithink supports 10 gigabits per
(42:56):
second,
Rob (42:57):
Not even close to enough.
PJ (42:59):
not, not a close to enough.
Not for VR
Rob (43:02):
14.
There are use cases for it.
Yes.
I mean, going back to CarMaxpoint, you could make.
With the available technology,you could use your phone as a
base station and have acertainly VR experience.
PJ (43:15):
Yes.
Yeah.
Rob (43:16):
You could potentially do an AR experience with lower
resolution cameras and fewer ofthem.
Again, what is the use case?
Do we need what it has?
I think carmack hit the nailsquarely on the head where a, an
apple headset that just fed fromyour phone.
Would sell a lot more than a$3,500 headset that has no use
(43:38):
case.
Right now,
PJ (43:39):
Well, what I'm thinking very specifically about is the media
use case where if we're reallytalking about, I wanna watch a
movie on a plane.
I don't need any cameras forthat.
Fundamentally, I don't needcameras looking at my eyes.
I don't need cameras.
Looking at the outside world.
I just need displays, and if Icould just pop that under my
phone and then watch whatever Iwanted, like, okay.
(44:03):
That covers like the media usecase that everyone's talking
about right
Rob (44:07):
That is Carmack's argument right there.
That is it.
You can do the Killing app thatwe have right now for a lot less
and a lot easier than what we'recurrently doing it, dad, we can
do it for it.
Significant cost reduction andget that same movie experience.
All we need for that movieexperience is the display and
(44:29):
potentially the head tracking.
If there's, uh, any 3D componentto it where you can kind of move
your head around or interactwith the scene.
All that's cheap and easilydoable and would be perfectly
doable from an iPhone.
Drive such a display.
Why they don't do that?
I don't know.
I don't think all the extrahardware adds much to the use
case because that is the appthat is the only use case that I
(44:51):
can see the Vision Pro having.
And we can do it much, muchcheaper and I think others will.
It Won't have the Appleecosystem to back it.
And that may be ultimately thekiller feature is the Apple
ecosystem and not any of theselittle Tech demo apps.
PJ (45:08):
I think this is gonna be the big question, is the ecosystem
enough to basically overpowerall of these obstacles that
we've talked about for ar, andit's gonna be an interesting
test case, frankly.
Rob (45:20):
Are you gonna buy one?
PJ (45:21):
I am not as of yet gonna buy one.
Uh, we will see, uh, I willadmit I do wanna play around in
this space.
So I did go the cheaper routeand I just got a Quest three on
order.
Rob (45:33):
Nice.
And I dunno, either, I don't ifwe're gonna buy one, I don't
really fancy drop$3,500 on oneof these things and hopefully
maybe they give one.
I don't know.
We'll, I'll, I'll prod around,see if I can
PJ (45:46):
See if we can get one.
That'd be, I think it'd be funto do like some sort of bake off
between the two making it likethe ecosystem be the
centerpiece.
Like, is, is this enough?
Like, I think that's a reallyvalid thing.
Well, I have a feeling we'regonna be returning to the
subject.
At least once or twice more thisyear as we see the results of
(46:09):
these devices.
Are they catching on?
Are they not?
Rob (46:11):
Yeah, I can't wait to see what people say.
It's gonna be real fun to seehow it pans out and see how pan
my technology gets.
PJ (46:18):
At the end of this, I think there's a two B continued dot,
dot dot.
Rob (46:23):
I think so.
Popular Podcasts
Las Culturistas with Matt Rogers and Bowen Yang
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com