February 26, 2025 • 53 mins
Discover the fascinating world of virtual reality storytelling with our special guest, Matthew Celia, the creative director at LightSail VR. Through an engaging conversation, we promise you’ll gain unique insights into how Matthew’s journey from film school to pioneering VR storytelling has shaped the narrative landscape, where story takes precedence over technology. We highlight the remarkable work on projects like "Paranormal Activity: The Ghost Dimension," illustrating the synergy between technical innovation and compelling storytelling that defines LightSail VR’s reputation.
Explore the evolution, challenges, and triumphs of VR storytelling as we unpack the lessons learned from early VR camera placement errors and the transition to 180-degree formats, which elevate viewer engagement. By simulating the experience of a live concert, VR storytelling grants audiences the power to engage actively with narratives, commanding their complete attention. We praise the VR series "The Faceless Lady" for its captivating performances, which showcase the unmatched power of acting to enhance the immersive experience in a virtual world.
Venture into the technical heart of VR production as we tackle the intricacies of sound, image, and VFX. Hear how spatial sound and stereoscopic vision work hand in hand with cutting-edge tools like the Canon Dual Fisheye lens to bring depth and precision to VR experiences. We address the VFX challenges uniquely posed by VR, where every shot demands meticulous planning and correction. Drawing inspiration from classic cinema, we discuss the art of mastering camera movement in VR, ensuring comfort while maintaining engagement for the audience. Join us on this journey as we celebrate the thrilling intersection of VR and entertainment.
Hosted by Foxtrot X-Ray’s founder and “chief pixel pusher” Paul DeNigris, who brings to the conversation 30 years of experience in both independent filmmaking and visual effects, as well as 20 years of experience in teaching all aspects of digital filmmaking at the university level.
For episodes, transcripts, and more, visit http://vfxforindies.com
For more information about what Foxtrot X-Ray can do for your film, visit https://foxtrotxray.com
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Paul (00:00):
Step inside Kilok Castle, a magnificent stone fortress deep in
the Irish countryside and home to LadyMargaret, a tragic spirit doomed to
walk the crumbling manor for eternity.
Three couples compete to survive theghost's sadistic games in The Faceless
Lady, an immersive stereo VR experiencefrom Meta, Crypt TV, and Eli Roth.
This week on VFX for Indies.
(00:41):
Welcome to VFX for Indies, a podcastabout the intersection of visual
effects and independent film.
I'm your host, Paul DeNigris, visualeffects artist, filmmaker, and CEO
of boutique VFX shop, Foxtrot X-Ray.
Joining me on this episode todayis Matthew Celia, creative director
and managing partner at LightSailVR, A bleeding edge studio committed
(01:02):
to pushing the envelope in termsof storytelling in virtual reality.
Welcome to the podcast, Matt.
Matt Celia (01:07):
Hey, thanks for having me, Paul.
Good to be here.
Paul De (01:10):
So on this podcast, our goal is to educate filmmakers, usually
independent filmmakers who aren't supertechnical about aspects of visual effects.
And in this case, virtual reality.
So if you like what we're doing here,viewers, uh, if you like what we're
doing here, please like, and subscribe.
So you don't miss any of our newepisodes and you can check out our
back catalog on VFX for indies.
(01:33):
com.
My company Foxtrot X-Ray had theopportunity to collaborate with
Matt and the team at LightSail VR.
On The Faceless Lady, which Iteased at the top of the show.
We'll get to that in a second beforewe do Matt, just tell us a little
bit about yourself and your career.
It seems like you've been inthe VR space for a long time.
Matt Celia (01:49):
Yeah.
So thanks.
Um, I'm a, I'm a film school kid,you know, went to film school
back in 2003 Chapman University.
That's where I met my nowbusiness partner, Robert Watts.
Um, and you know, always was interestedin directing and creating content, but
always had a kind of a technical edge.
When I graduated, I went right intolike post production, working for Warner
(02:12):
Brothers, doing digital first content.
I ended up doing some commercials,working at a commercial
production company for a while.
And, uh, eventually landed into creatingmy own documentary, shopping that
around, and, uh, getting that hardreality of struggling to break into
the 2D, very competitive landscape.
(02:32):
Um, at that time, Robert, my businesspartner, he, handed me a piece of
cardboard with a phone inside it andsaid, Hey, I think this is like 2015.
He's like, Hey, I think thisis going to be big one day.
You should, you're technical.
You should take a look.
And you know, like creatively, I've alwayshad a artistic theater style to my stuff.
(02:52):
It's very real in the moment.
It doesn't really rely on a lot ofovert visual effects, I would say.
I mean, there's always likea lot of hidden stuff, but.
Yeah.
You know, I like to be in themoment and be in the real present.
And when I saw the piecewas called Walking New York.
Uh, it was by Chris Milk'scompany at the time.
And it was just like, I was in NewYork and I was in a helicopter and
(03:14):
it was just letting the scene playout and it felt like theater and it
really ignited a passion in the kindof storytelling that I wanted to do.
So I spent the next, like,48 hours watching furiously
everything I could find.
And I went down to VRLA, whichis a big conference here for VR.
And it was a bunch of hackedscience fair projects.
(03:37):
And everything I sawwas super interesting.
But what I noticed was everything Isaw was very much about technology.
And not about storytelling.
It was very much like, Hey, I puta camera down, isn't that cool?
But there wasn't that sense ofcharacter, that sense of conflict,
the sense of emotional stakesthat you get in a good story.
(04:00):
So I called Robert up and I said, Hey,I think as an artist, someone who's been
struggling to find how they fit into thisworld, I have something to say finally.
And I think this is it.
So we bought like 10 GoPros whenwe started making content and
people saw us on social media and.
We ended up creating one of the firstpieces for Paramount Pictures called, uh,
(04:21):
Paranormal Activity The Ghost Dimension.
It was a in universe original contentwritten by the writers of The Ghost
Dimension that, uh, tracks like thesethree kids summoning a ghost in a garage.
And it was one of the first VR piecesthat we were That had real people acting
scared had a mechanism that was, youknow, in universe, you know, we used all
(04:46):
the paranormal activity like blue and thefuzzy cameras fitzing out like really dove
into the world and the rules of the world.
And it resonated with folks.
Uh, got 10 million views.
Um, and we, we would walk around atconferences with like, uh, Gear VR in our
backpack and like hand it to people andlike they'd watch this horror content.
(05:06):
It was so wild.
Um, but they were like, no, that was cool.
I feel like I'm emotionally gripped by it.
And that was the thing for us that saidlike, Hey, we can take the a hundred
years of storytelling and put it intothis new medium and we can start to
figure out how to translate that.
And that is the like ethos of LightSailVR is we're story first people.
(05:28):
We're tech heads.
We're nerds.
Um, but at the end of the day, we workwith directors and creatives who want
to get into the virtual reality worldand experiment with this new medium.
And we translate how that actually works.
Um, and that's been whatwe've been doing since 2015.
So almost 10 years now,it's a pretty wild ride.
Paul DeNigris (05:51):
That's great.
Yeah.
I, you know, that was actually, my nextquestion was going to be about this
phrase story first, cause it's all overyour social media and your website.
Like it's, it's your mantra.
Obviously
Matt Celia (06:01):
it's, it's, it's a finding aspect of what we do.
And, you know, Robert and I.
We could do a lot of stuffand we've done a ton of stuff.
We've been very flexible as a company.
You know, we've done everything fromtraining to, to, uh, entertainment,
like The Faceless Lady, but ourheart is in entertainment and our
heart is in storytelling and theunique thing that I think LightSail
offers to a lot of productions isthat we speak both sides of that.
(06:25):
That chasm.
We are very technical, but wealso understand like the needs of
creating characters that emotionallygrip you and that puts you there.
And our phrase here is thatimmersive is not just about
the physical bounds of reality.
It's an emotional immersion as well.
And you can't forget that because youcan be immersed in a 2D film as much
(06:45):
as you can be immersed in a VR project.
It's about how good your story is.
Paul De (06:50):
So what are some of the things about VR that you think really
lend themselves to, to storytelling?
You know, what, what is it, whatis that secret sauce that you, that
you guys have discovered that makesstories compelling in the VR space?
Matt Celia (07:02):
Yeah, if I had to distill it down to one word, I
think that word would be presence.
It's the idea of feelinglike you're actually there.
Um, and if you start to think of all yourcreative decisions, As subservient to
that word, and that's your North Star,good things, I think, start to happen.
So, it's about feeling like you'restanding in the best spot in the
(07:26):
room, where you would naturally be.
A lot of early VR creators wouldput the camera in the dead middle
of the room, thinking like, Oh,that way I can see everything all
around me, because 360 at the time.
And if you actually go to a partyand you look at where people stand,
Nobody stands in the middle of a room.
Like, that just doesn't happen.
And so understanding, like, where a goodvantage point is helps feed that reality.
(07:50):
Like, how high a camera is.
That feeds that reality.
How people interact with the camera.
Feeds that reality.
And so, the idea is to make people feellike they are there because the greatest
advantage of being in VR Is that youlook around and you are in the story
and you want to give the audience thatagency, that choice to be able to choose.
(08:14):
I'm going to look at thepainting on the wall over here.
I'm going to look at, you know, thischaracter over here because that choice.
There's something, I don't know, there'ssome neurological thing about the way we
are built as human beings is where whenyou make that choice, when you initiate
that action, the connection you have tothe material is somehow stronger, and
(08:35):
this is backed up by dozens and dozensof studies in this field, and it's
why virtual reality is such a powerfultraining tool and a powerful, um, like
therapy tool, um, is because there issomething very real about you saying,
I looked at that, I looked at that.
That makes it a more active viewingexperience rather than when you watch
(08:58):
TV on your couch and you veg out, it'svery passive and things just fly by you.
That's okay too.
But VR, I think naturally asksyou to lean in a little bit.
Paul DeNigris (09:09):
Yeah.
I mean, it sounds like in a wayyou're giving a little bit of
directorial control to the viewer,kind of making them a partner, right?
You're, you're deciding this is thebest vantage point, but then the,
the viewer ultimately has the choiceof what they're looking at, right?
Yeah.
Matt Celia (09:23):
And, and, you know, VR has gone through,
um, VR is a big topic, right?
There's interactiveVR, there's passive VR.
When we started, we were doingeverything in 360 degrees.
And that was like.
People really had agency becauseyou could look behind you.
You could look anywhere and directingthat was a challenge because you
really had to think a lot about, allright, what are people going to do?
(09:47):
How are we going to get people to lookat the thing I want them to look at?
Now we've kind of evolved tomore of a 180 format for a lot of
narrative content, which I think is.
the best medium of this.
And you sacrifice a little bit ofthat immersion because obviously when
you look to your left, you see like ablack line that goes into nothingness.
(10:10):
But you know, as a creative, asa storyteller, I'm still saying,
Hey, generally look this way.
I'm generally framing my shots.
I can cut faster.
I know that you're going to belooking that way, but there's still
such a wide canvas that you stillhave that agency That it's the same
kind of agency you have when youwatch a concert or a theater show.
(10:34):
And you know, it's like that samekind of reason, like when you watch
a concert on television and they'recutting shot to shot to shot, it's cool.
That's one kind of the experience, butwould anyone say that that replaces
the value of going to a live concert?
I don't think so.
And I think it's because at a liveconcert, you're making that conscious
choice to look at the guitar player,look at the drummer, look at the singer.
(10:55):
Um, and it, it createsthat same kind of agency.
Paul DeNigris (11:00):
Uh, although most people are, most people
are looking at their phones.
That's true.
Which is a whole different thing.
That's one of the nice things aboutVR is it's hard to look at your phone
when you're, when you're consuming VR.
Matt Celia (11:12):
I mean, I think it's a great medium for, for creatives.
Yeah.
Like.
Think about it.
Right now, a lot of directors, theycreate content for phones or for,
for TV even, and they're watched inthe worst circumstances on earth.
Like, there's people talking, there'stexting, you're like making dinner, my
wife's folding laundry or doing whatever,like I'm trying to watch something.
(11:34):
When you're in VR, everything's gone.
The only thing that you arelooking at is the stuff that
the creator put in front of you.
And that is tremendously powerful, but atthe same time, I think that's a really big
responsibility because it does mean that.
If you can't keep people'sengagement like that, then you
(11:54):
have serious story problems.
Paul DeNigris (11:57):
Yeah, for sure.
And a lot of it, I think alsocomes down to the acting, right?
The acting has to be, has tobe engaging and naturalistic.
There's a, there's a moment in, uh,in The Faceless Lady that totally
sold me on the, on it as a show.
Uh, it's when, um, the, the, the,the lead character Ella, right.
(12:17):
It's when Ella is sitting on her bed.
In the room and she's she hasthis this emotional moment.
She's crying and we are we are sittinglike as if we are, you know, kneeling
on the floor in front of her likewe would if if we were trying to
comfort her in in that situation.
Right?
Yeah.
And it's like, okay, I'm I'm I'mtotally the In this world and on
(12:42):
this journey with these folks.
And then when, when some of the charactersdie and you, you guys put the camera
like right on the chest of the dude who'sdying and you watch him take his last
breath in VR and it's like, okay, that'sa, that's a real visceral experience.
Yes, I can look all over the place,but I don't want to because the acting
is so riveting and drawing me to it.
Matt Celia (13:04):
And I will say, like, John Ross, the director of
The Faceless Lady, was, is justa master of moving the camera.
And he really, we worked really closelytogether and prepped for that production.
We did a bunch of camera tests.
We did different heights.
We did different distances.
We did different angles.
And we put it all in a headset to tryto build the visual language of the
(13:26):
show, to know how far we could push,what would work, what would not work.
And a lot of those moves are theresult of many, many takes to get
it just perfectly dialed in, right?
To feel comfortable in VR.
Because there's this whole comfort feelingwhen you're moving the camera as well.
And I just think he did a reallygood job of Of listening and watching
(13:47):
references and thinking carefully aboutthe vantage points and the feelings
he wants to elicit out of audiences.
Like there are a number oftimes in the show where he does
something that does make you feela little like, Oh, I don't know.
Oh, that's a little weird.
You know?
And he used it not just because,Oh, this is a cool shot.
He was very specific about it.
(14:08):
He's like, no, I wantsomeone to feel off balance.
Their friend just died.
We should be feeling off balance.
And I love that he did that becauseI think that that is using the
medium to its fullest potential.
Paul DeNigris (14:20):
Yeah.
It's another dimension we don't havein, in 2d, uh, filmmaking at all.
So you've mentioned a couple oftimes, you've said the, you've said
360, you've said 180, uh, you know,for, for the, the folks out there
who maybe are not familiar with...
super well versed in VR.
Can you just talk like, you know,we've got 360, we've got 180, we've
(14:40):
got mono, we've got stereo, we'vegot passive, we've got interactive.
Could you just give us sort ofthe menu of, of the different,
uh, options out there?
Matt Celia (14:48):
So VR, like when you say VR, like think of VR,
like you think of television.
It is a very big and vast open medium.
Basically VR is anything consumedinside a virtual reality headset.
Um, so the kind of content that LightSailis most known for is passive, linear,
immersive video is kind of what I call it.
(15:09):
Um, and immersive video comesin a variety of flavors.
The first is 360, which is 360 degrees.
And that is usually captured with manycameras looking every single direction.
All around.
And so you can look up, down,left, right, backwards, forwards,
and you feel completely immersed.
It's like Google Street View,but like, you know, a video.
(15:32):
Um, those flavors also come in 2D,monoscopic, and 3D, stereoscopic formats.
Um, so, 180.
Is just the front half of that sphere,and typically 180 will most always be
stereoscopic because it's easier todo, and it really adds the immersion.
(15:56):
Um, but of course you can make itmonoscopic formats as well, and when
I say monoscopic and stereoscopic,that's like if you're watching, you
know, a 3D movie, because like you'rewearing a headset on your face, so you
might as well take the advantage thatyou have a lens of two different eyes.
And get that depth, because that depthcan be really exciting for storytelling.
And also, since you're working in aformat that doesn't have depth of field.
(16:20):
Right.
You don't have shallow depth offield and like a blurry background
playing with depth in a stereo senseis a really powerful storytelling
tool to enable you to say, thisis what I want to focus on.
You know, this is the core of my scene.
Paul DeNigris (16:37):
Gotcha.
So, so putting the most important thingsort of at that convergence point is a,
is a term of phrase that we used a lot.
Yeah.
Matt Celia (16:46):
Yeah, exactly.
And we don't, you know, a typical flat.
3D movie will move that convergencepoint to be what screen space is because
that's the most like the focal thingin VR It's slightly different because
we do it more like how you see whereasyour convergence point You're is way
in the distance because you don't seestereo depth past like 30 40 feet And
(17:10):
then everything comes out towards you.
So you only get that.
Um, I believe it's negativeparallax come towards you.
So you only really get that.
Um, but you know, it's about placingyour subjects in this like sweet
spot zone between like three and 15feet where things look particularly
good and sharp and accurate.
And that's like, kind of like in life.
(17:32):
When I walk up to you in the real world,I'm not standing 20 feet away from you.
I'm also not standing one foot from you.
I'm like, you know, thatthree to like six feet
Paul DeNigris (17:41):
realm, you know?
Right.
If you're six inches from the personyou're having a conversation with,
it gets a little uncomfortable.
Matt Celia (17:46):
It gets a little uncomfortable.
Paul DeNig (17:47):
Both in VR and in...
Matt Celia (17:49):
real life! And that goes back to, like,
creatively, thinking of the cameralike a person, and like, thinking about
where is your presence in the space.
Paul DeNi (17:59):
Uh, I imagine the use of sound is also really important to
help direct the viewer's attention.
Matt Celi (18:05):
I mean, less than you would think, to be quite honest.
Like, it's definitely a commonthing to say, like, use sound
to direct people's attention.
But the reality is, is that, um,unless you back it up with visual,
like, something visually that, like,Says, Oh, that is what I'm seeing.
There's so much going on.
There's so much sensory overloadthat it gets overlooked.
(18:27):
And like, that's a really importantpoint to bring up is that you have to
understand when someone put this headseton their face, especially they're new
to it, and they've got this enormouscanvas to look at, and they've got this
spatial sound and they've got the 3d,that is a lot of sensory information.
And it can be absolutelyoverwhelming to be in that.
(18:49):
Um, so you have to take it slowand you've kind of got to hit the
nail on the head a little bit.
Um, spatial sound is huge.
It's that thing that adds to presence.
And I would say it's almost more importantin VR than it is in a flat film, because
we always hear 360 degrees always.
(19:10):
And so having thatelement of it really does.
You know, tell your nervoussystem, Oh, I'm actually here.
Paul DeNigris (19:19):
Yeah, for sure.
Uh, particularly like the battle scenesat the beginning of The Faceless Lady.
Yeah.
You're hearing this, this full.
You know, world of these guysfighting and, you know, all of
the stuff, the fire and the swordsclanging and things like that.
And it, uh, and it really puts youin it in a way that, yeah, watching,
(19:46):
you know, with your soundbar underyour television, as good as that
may be, it's still not, you know,being beamed right into your ears.
Matt Celia (19:52):
And it's, um, it's head tracked, right.
I think people don't alwaysunderstand that it's actually,
it's It's real spatial.
So if I'm hearing a character talkit right in front of me and I turn
my head to the left, now I'm hearingit just in this ear, as if they are
still just right in front of me.
There's actually like a spatial level.
Anchoring of all the sound objects.
(20:15):
And that's the thing that reallyelevates it and makes that feel
like, Oh, I'm actually here.
Paul DeNig (20:20):
Uh, so you mentioned monoscopic and stereoscopic, so
one eye versus two eyes, right?
So, uh, just really quickly for, forthe non technical folks, the idea is
we see, we see two slightly differentimages, you know, one in each eye.
Normally, right?
If you've got, we've got stereoscopicvision, so we're seeing a little bit of
(20:41):
separation and that separation is whatour brain turns into three dimensions
and turns it to depth perception, right?
So stereo, we're taking advantage of that.
We've got a left eye imageand a right eye image.
Matt Celia (20:53):
Right.
And the cameras we use, the lenswe use specifically on The Faceless
Lady is the Canon dual fisheye, whichreally has a lens for this eye and a
lens for this eye, and that's what'scapturing onto a single sensor.
So you do get about 60 millimetersworth of, uh, parallax on that.
Yeah.
Which is, which is the averagefor, for, It's an average for
(21:13):
human, for human eyesight.
Everyone's between like 55 and 72.
Paul De (21:18):
As opposed to something like, uh, like avatar where they were
shooting with two complete camera rigsand this whole, you know, uh, complex
contraption to put the two lenses tohave two cameras and have one shooting
down into a mirror so that they cancreate the separation and all that.
And they can dial the separation.
Matt Celia (21:37):
You can shoot VR just like that.
But the problem becomes when yourfield of view is 180 degrees up.
Down left and right it, you would seethe other camera system right above
you, you know, and, and that becomesa huge rig and very complicated.
Paul DeNigris (21:52):
So, so that, that dual fisheye lens really has
kind of democratized the process.
Matt Celia (21:58):
It's been the biggest, uh, game changer.
I think when Canon showed up withthat lens and it's not just a
dual fisheye lens of like, it'sa really good dual fisheye lens.
Like it's an L series glass.
It's.
Amazingly sharp.
It's got very, very lowchromatic aberration.
It's got low flare.
(22:20):
It's got, um, very low distortion values.
So it's, it's It's a very highquality precision instrument.
And I think Canon did an incredible featof engineering with it, to be honest.
Paul (22:36):
Um, they certainly that's lens certainly makes the compositor's
job harder because we can't, we can't,you know, throw a couple of pixels
of blur on something to hide, hidethe seams, you know, when everything
in the frame is super sharp, um, youknow, everything that we added to
the frame had to be Equally sharp.
And there was no, a lot of the cheats thatwe do, uh, in, in regular 2d compositing,
(23:01):
um, yeah, VR is not available to us.
Matt Celia (23:04):
VR is just very unforgiving for visual effects.
You know, also you're takingwhat you do and you're blowing it
up like so large onto a screen.
So, and, but when you're working on it,you're working in it on a small computer
and it's like kind of tiny and it's,it's just, it's, it's the challenge.
Paul DeNigris (23:24):
Yeah.
So, you know, for, for those who arenot, not experienced with this, that
dual fish eye is going to shoot two.
Uh, two 4k images side by sideand they are spherical, right?
So they, they're not flatlines are not straight, right?
The, the, the photo, the pictureshanging on, on Matt's wall
behind him would all be curved.
(23:45):
All of those lines would be curved, right?
So now we have to, we're eitherworking in that spherical space and
making sure that everything we do isrespecting that curve, or we're taking.
A little section that we're going towork on and taking that curve away,
straightening it out, doing our work,and then putting the curve back on
and sticking it back in the frame.
So it adds a lot of complexity and alot of like these sort of geometric
(24:09):
gymnastics that we have to do to get toget everything to sit right in the world.
And then we're doing it in two eyes.
Matt Celia (24:16):
Right.
So what you do has to be exactlythe same on each eye, except
for your horizontal disparity.
Right?
And that's that's a huge challengebecause, you know, you're cutting the
character out right with some roto.
Well, you have to do the exact sameroto on the other eye, and then
you have to offset that, and thenmake sure that that's perfect, and
(24:38):
if you don't, then you're goingto have slightly different rotos.
So like, we did a project where avisual effects team tried to convince
us that they could use AI rotoscoping,and I said, it's not going to work.
And they were like, yeah, it'll work,and they went ahead and did it, and it
did not work, because the AI couldn'tdo it the exact same on each eye.
So sure.
The left eye looked fine and theright eye looked fine, but together
(25:02):
they were slightly differentand they did not look good.
Paul DeNigris (25:04):
Yeah.
Yeah.
We, we encountered a number of timeswhere, uh, the Roto was the issue, right?
Cause you've got, we'd have, we'dhave, um, Ella's running at one point
and she's running past the camera.
And so the parallax was drasticallydifferent from one eye to the next.
And the details of her hair weredifferent from one eye to the other.
(25:26):
And when we, the Roto, the Roto teamgave us mats that were perfect, right?
If you looked at left eye.
Perfect.
Look at the right eye.
They were perfect.
But then you put them together and it waslike, something's, something's not right.
Yeah.
Something's wrong here.
It doesn't feel right.
You know, that was a note thatyou guys gave us a lot was this
(25:47):
doesn't feel right stereoscopically.
And it sort of drove me crazy becauseI'm like, I'm looking at, I'm like, This
pixel lines up to this pixel and thenit lines up to this pixel in this eye,
Matt Celia (25:56):
but then you look at it and
there's like, there's that disparityand you're like, is it too far forward?
Is it too far back?
Like, and there's always this like littlebit of like grid warping juice that we
had to, like, we had to do, um, some,some roof replacements and like patches
took for ever because the 3d roof wasnot a hundred percent like aligned with
(26:18):
like the lens and we just had to like.
Sub pixel warp it, whichis challenging to do.
Uh, on huge footage files.
Paul DeNigris (26:28):
Um, and you guys, uh, you guys work in,
uh, resolve in fusion, correct?
We work in resolve, in fusion,um, primarily for financial reasons.
Um, just because it's, it offerspretty much the whole tool set we
need at a dramatically reduced price.
But, you know, on The FacelessLady, the thing that we explored
(26:50):
that was really powerful was theintegration with the actual timeline.
Cause we didn't have to do the visualeffects process of having to bake
plates out and export plates andcheck those and do all that stuff.
And artists could open up the timeline,select his or her clips, right
click, go create fusion composition.
And they had everything theyneeded in context right there.
(27:13):
Um, and that was, thatwas pretty powerful.
Um, saved us a little bit of time.
Yeah, that's gotta be really helpful.
How is, uh, fusion in terms of.
Being able to monitor yourwork in a headset in real time.
Matt Celia (27:25):
It's tough.
It does have support toplug a VR headset into it.
Um, but we've created a tool calledvr.ndi at LightSail, which uses NDI to
sling pixels and put it in a VR headset.
And so because we're a distributed shopand everybody dials in from wherever in
the world that they're working, all thecomputers in the closet have a plugin
(27:46):
called Nobe display and Nobe displaysends and eat NDI signal out of resolve.
And we use NDI bridge, which is freeby the way, to sling that back to the
artist's home computer, where theyhave a headset, uh, where that home
computer can receive that signal and thenbroadcast to their, their quest headset.
(28:06):
So they're able to seestuff it's compressed.
It's not always a hundred.
Like if you're doing the sub pixelwork, they downloaded it to their
local machine and did that sothey can have full res quality.
Um, but if you're just placingstereoscopic titles and you're just
like getting into the right space.
It's enough.
Paul DeNigr (28:25):
Yeah, that's great.
We, in my shop, we work in Nuke andNuke has the ability to feed a VR
headset, but it's not real time.
And you can't see your mouse andyou're in VR, you can't see your, you
know, your mouse cursor or anything.
So we, so it would be,you know, headset on.
Right.
And you set your perimeter.
So you could lean forward and you'dbe able to see through the world.
(28:48):
Yeah.
I'd be like.
Lean forward, tweak something,lean back, wait for it to refresh.
Matt Ce (28:53):
Yeah, or you do, or you do that thing where you take your
headset like on and off all the time.
You're like, uh, what is this?
What is this?
Um, I use the, um, I use the Quest Prowhen I'm at my desk and you can see
like it doesn't have the bottom bit.
And like, that's killer for me becauseI can see my keyboard and see my mouse.
Paul (29:09):
Oh, that's, that's great.
Yeah.
For me, the, uh, thechallenge was I wear glasses.
Yes.
So, so I would, that it was justkilling the bridge of my nose, taking
the head, the headset on and off.
And then I got the, uh, Zenni eyeweardoes the, uh, we're not sponsored by
Zenni, by the way, but Zenni, if you'relistening, I would love a sponsorship.
I love your glasses.
Uh, Zenni has the, uh, theinserts for the, for the quest.
(29:32):
Um, So I got those in my prescriptionand put those in and that was great.
But then I couldn't do the on off becauseI'm totally blind with no glasses.
So I'd have the headseton and could see clearly.
And then I take the, and I'mlooking at, you know, fuzz.
So
Mat (29:46):
I used to have glasses and, uh, it took working in VR for like seven
or eight years for me to finally go like.
I think I'm going to get the surgery.
I think I'm going to do theLASIK and, uh, it was worth it.
Paul DeNigris (30:00):
Yeah, smart move.
So in a typical show, um, and, and Iknow there's probably no such thing
as a typical show because every showis kind of, but, but typically how,
how is VFX fitting into your projects?
You know, how are you using the effects?
Matt Celia (30:19):
Here's the, the, the reality of VR filmmaking is every
single shot is a visual effect shot.
Every single one.
Because at the bare minimum, you're takingyour fisheye lenses and you're applying,
you know, an ST map or doing some sort ofunwrapping to get it into equirectangular
space and you're reconverging yourstereo and so on and so forth.
And then typically you'llhave stuff that needs cleanup.
(30:42):
Shadows, tripods, youknow, like gack, whatever.
Like there's, there's, uh, VR is a lotabout what's invisible visual effects.
And I think this has been talkedabout a lot, just in the broader
visual effects community.
Cause you have a lot ofproductions coming out saying like
we did it all practically.
There's no visual effects.
(31:03):
And you're like, whatare you talking about?
There's like 2, 000 visual effects shotsjust to clean up like, you know, just
to do a merge or a comp or take outa shadow or take out all this stuff.
And so.
Visual effects in VR is really aboutmaking the visible invisible, because
again, going back to that word presence,you want to maintain the idea of presence.
(31:28):
And if you're seeing a camera shadowright in front of you, because the
sun's behind you when you shot thescene, well, that's pretty distracting
because you're looking at a camerashadow instead of just the ground.
Um, and so there's a lot of that.
You know, I think we were reallyfortunate on The Faceless Lady to have the
opportunity to collaborate with you folksto do creative visual effects, because
(31:51):
that's where the real fun stuff happens.
But to be honest, because,you know, VR productions are
usually budgetarily challenged.
Um, sometimes you don't have, youknow, the coin to get even have fun.
Um, you're just, you know, spendingit all on taking out lights.
Paul DeNigris (32:10):
Yeah.
So, so VFX is.
Part and parcel of thewhole process, right?
It really is.
Okay.
So, so therefore VR creates, youknow, typically what I ask a, a
guest is what are some challenges andproblems that VFX helped you solve?
You, you just covered that, right?
Yeah.
I mean, effects is, is helping you guysall the time just to, just to make VR.
Matt Cel (32:33):
If you want to make VR before you go make anything in VR, the
key thing is you should take like abeginner intro visual effects class.
Just to understand thebasics of how stuff works.
You should understand,like, what a clean plate is.
You should understand howgreen screen actually works.
You should understandhow compositing works.
Like, you don't need to be anexpert artist and any of that stuff.
(32:55):
Like, lord knows I am not, like,the world's best roto artist,
or composite keyer, or whatever.
But I do understand the basics, sothat when I'm on set, I know, like,
hey, I need clean pixels there.
And if I don't get those cleanpixels, it's going to be a
really expensive cleanup job.
Because if you think about it, youhave 180 degrees field of view.
(33:16):
If you want cinematic lighting, you'regoing to need to put lights in your frame.
Like, or you're going to need to spend alot of money on very large sources, but
lighting with super large sources faraway from the talent means that you're
not going to get the look you want.
So you'll always paint out lights.
Well, now you want to move the camera.
(33:37):
Okay.
Well, that just became a lot harderbecause how do you capture a clean plate
and move the camera and do that paint?
That becomes really challenging.
Oh, you want your light over here?
Well, guess what?
Your whole camera shadow isnow on this opposite wall.
How are you going to clean that up?
And so, it is a frustrating,like, like, jigsaw puzzle.
(34:01):
Like, you just, you know what it is?
It's kind of like you're in the, you know,submarine and a pipe bursts and you put
your hand on it and then another thingbursts and you gotta go put your hand
on that and then like, and like, that isAll these things in visual effects is the
answer to solve all of these problems.
But if you, um, if you don't shootit correctly or think about it,
well, you know, every hand thatyou're covering those pipe is dollars
(34:24):
and your visual effects budgetcan quickly spiral out of control.
Paul DeNigris (34:28):
For sure.
And then on the flip side, havingto shoot for VFX creates a whole
other set of problems, right?
Because like you said, youhave to shoot clean plates.
You have to shoot greenscreens a particular way.
Uh, and sometimes it'splaying whack a mole.
It's this, this, thesame, the same metaphor.
I remember like, uh, we had thosedrone shots in the, in the hedge maze.
(34:52):
And we go back and forth and I'd fixone thing and you'd be like, here,
this is what I want you to fix.
And I'd fix that in the process offixing that I've knocked this other part
of the shot completely out of whack.
And you're like, well, that wasgood, like three versions ago.
Now it's all fucked up and,
Matt Ce (35:07):
and, and, and it's hard because as you're doing it, you
just version, version, version.
I mean, we go through stuff so oftenand I will say this too, in VR, because
there's such a huge frame of stuff.
Just QCing your own work ischallenging because you'll be
looking at this one part of the frameand be like, yeah, that's great.
(35:27):
That's awesome.
And you'll watch it four or fivetimes and you'll totally forget
about this part of the frame.
So, you know, if I were to distill thisconversation down to one sound bite
of advice is just give yourself moretime, more time to capture on set to
get the assets, more time in post toQC it, to watch it because every part
of the process has friction and there'sjust so much more that you're creating.
(35:51):
And you're working, you'reworking at 8K resolution.
It's huge.
You're working at 60 frames a second.
That's almost three X a normal film.
That's that's a lot of data.
And so, and then your frame is enormous.
So it just takes time.
It's not that it's so muchharder as much as it is.
(36:13):
It just takes more time.
Paul D (36:16):
Yeah, it takes more time to send the files to a, to a vendor,
to, you know, to download even withinyour internal team, you know, offsite
having to download stuff for, for,
Ma (36:26):
I mean, that's primarily why at LightSail, we centralized everything.
Paul DeNigris (36:30):
Yeah.
Matt Ce (36:30):
And we have people dial into our computers because we were
spending days just copying files to ahard drive and then shipping it off.
And then, oh, wait, therewas a problem with that.
Oh, well, I got to do that again.
You know, their drive and like.
You start to lose a lot of time.
Paul DeNig (36:45):
Yeah, I can imagine.
So you talked about, uh, John Ross,the director of a faceless lady
and his uh, facility with movingthe camera planning camera moves.
Can you talk about, you know, likewhat are, what are some of the,
the camera rigs that you guys used?
Are we, are we, are we steady cam?
Are we Dolly?
Are we on cranes?
Are we on drones?
You know, how are wegetting these cameras?
Matt Celia (37:07):
So, One of the big things that you have
to think about with VR is you needit to be as stable as possible.
Um, if you're, if you're having multipleaxes of rotation, it's a good way
to make your audience sick, right?
You want to let the, the audience bein control of pan, tilt, and roll.
(37:31):
You want to controlthings like forward dolly.
Lateral dolly, you know, you want tocontrol some of the bigger elements,
but you don't want your camera justdoing like a little bit of this
because they're going to feel liketheir whole world like doing this.
So I will say that, youknow, we stabilized over 100
shots in The Faceless Lady.
(37:52):
Using Syntheyes.
And it was a large portion of the workthat we did, because even when you're
on a Ronin, like a good old Ronin rigthat has like the, the skeleton arms
and is like a really good operator,um, there's still like a little bit
of that shake and a little bit of thatvibration and a little bit of the forward
and backward and six axis movements.
(38:13):
And that can be really, reallydisturbing in VR because you kind
of feel like you're doing this.
Even though your body is not actuallymoving and that separation is what creates
simulator sickness That's the one wordfor it, but really it's your like your
nervous system saying like something'snot right here So I should feel sick So
(38:34):
when we think about moving in camera,we typically stay on dollies because
they're rock solid dollies of courseare really challenging because You'll
see the track right underneath you.
So now again, you gotto paint out that track.
So what we'll do is we'll, we'llput the dolly on a long jib arm and
that'll be kind of what we rock.
(38:55):
So that way we just need more space,but we at least can pull backwards
and not see the edge of the track.
Um, and that's a really effective wayto move the camera around, to be honest.
Uh, cause we are able to getthese like nice and subtle moves.
We're able to control thespeed, we're able to control
acceleration and deceleration.
So it's very, very soft, very,very linear as much as we can.
(39:20):
And a lot of that is why wehave headset preview on set.
You know, John is in the headset, I'm inthe headset, while we're capturing this,
and this is why we did so many takes.
Is like, we just had to get it just right.
And we had an amazing crew in Ireland.
Like we had just an incredible gripteam, incredible camera operators.
Um, but it, it is unforgivingto move the camera.
(39:45):
Um, and John moved it a lot, whichwas great, but we wanted to make sure
that every movement is so motivatedabout being in that right spot because.
You have to remember, even whenyour camera's static, it'll still
feel like it's moving becauseyour audience is looking around.
So they're doing their pans,they're doing their tilts.
Paul DeN (40:07):
Yeah, it's really, the movement's really effective, especially in
the, in the hedge maze sequence, you know,because, because you're in, you're in that
space and you've got the hedge walls oneither side of you and the cameras pushing
down the, the, the, the row or, or pullingback and it's, and it's going by, did
you get, were those dolly shots as well?
Or was that...
Matt Celia (40:26):
so the hedge maze was not dolly shots.
The hedge maze was on a Ronin withlike these like skeletal arms and
a Rua running through the maze.
You know, trying to keepit as steady as possible.
We did a lot of post stabilizationon that stuff to clean it up and make
it look as comfortable as possible.
There was moments in that episode though,where it's definitely not comfortable.
(40:48):
And that's the conversation that Johnand I would go back and forth with a lot.
We'd say like, Hey, you know, we canonly use this much of it in the cut.
And then we need to sandwich itbetween two comfortable shots.
Cause we like to approach VR aslike having a comfort budget.
Whereas you can have one uncomfortableshot, but if you have like three
uncomfortable shots in a row, guess what?
(41:10):
Your user is just, they'returning off the headset.
They're getting out of it.
And you don't want that.
You want to be able to use it to saylike, I'm being chased by a ghost.
I should feel a little uncomfortable.
Not like I feel queasy and nauseousand need to go lie down for an hour.
Yeah.
That's a great point.
I mean, movement is one ofthose things that I think.
People are very, in VR,they're very scared to try.
(41:34):
And they're also toldthat they can't do it.
And the reality is, the truthis it's somewhere in between.
And I think John Ross figured that out.
I think he found that, like,threading of the needle, where
he created the sense of movement.
And you know, interestinglyenough, he actually looked at
Citizen Kane as a touchstone.
(41:55):
Um, because when you look at thatmovie, Those cameras were enormous.
You're not like popping those on a littlelike gimbal and flying around places.
They're huge cameras.
So all your movement has tobe careful and controlled and
orchestrated like almost a dance.
And then also that film surprisinglyhas a tremendously deep depth of field.
(42:17):
Like you'll see all the way out the windowand he's like, you know, On his sled
as they're, as they're talking in theforeground and they're moving the camera
and the characters are moving towards it.
And him and I looked at thatfilm a lot and we like broke down
certain sequences about like, thisis how the staging should feel.
And that became a real touchstone forThe Faceless Lady that I think worked.
Paul D (42:39):
Well, there you have it.
All for all of you.
Citizen Kane haters out there who,who complained that it's constantly
the number one greatest movie ofall time on everybody's lists.
It's pretty good.
It's a pretty damn good textbook,even even now, even in VR in
technology that that Orson Welles.
Couldn't possibly have dreamt of.
Matt Celia (42:59):
But has a lot of the same challenges.
Like VR is about taking what's oldand making it new again, you know?
Um, because it is thebirth of a new medium.
It's not going to replace TV movies oranything like that, but I think it's
fundamentally going to be a differentway for audiences to experience stories.
And that's what I think is reallyexciting about it is that we start to
walk this line a little bit betweenstorytelling and story living.
Paul DeNigris (43:23):
And that gets really exciting being that our
target audience for this podcastand our established listeners
and viewers are indie filmmakers.
They don't have the budgets ofMeta / Crypt TV / Eli Roth, but
they want to make something in VR.
They've got a story thatthat screams out for VR.
(43:45):
And you said, step number one, take somebasic VFX training, learn, learn that.
Okay, what's the next step?
What, how, how does, howdoes somebody jump into this?
Matt Celia (43:57):
Yeah, it's a great question.
I mean, the first question you have toask yourself if you're creating something
in VR is, is my story better in VR?
And that sounds like a, a dumbthing to say, but you'd be shocked.
Um, you have to have a good reasonto want to put people there.
Like, I think Robert Zemeckis has anew film coming out, I think called
Home, where it's like a single shotin a house through ages of time.
(44:22):
And I look at that and I go,that should have been in VR.
Because, like That just feelslike so right for the medium.
Um, but a lot of folks comeinto our door and they say, I
want to make Hardcore Henry.
I'm like, Hardcore Henry is not in VR.
Like that's a first POV.
It's all about like movingthe camera and crazy stuff.
The, the key to start out is toget a camera and start looking at
(44:47):
content, you know, getting intothe ecosystem of VR does not have
to be tremendously expensive.
Um, a really good, still pro levelquality can be found with like
the Canon EOS R7 with uh, the 3.
5 millimeter dual fisheye lens they have.
It doesn't do full 180degrees, does like 150 degrees.
(45:11):
Um, it's not a full framesensor, it's a crop sensor.
It's not 8K, it's 4K.
But, that means that the files area lot easier for you to work with.
Like, because working in4K is tremendously easier.
Working at 30 frames a secondis tremendously easier.
And because it's not a full 180 degrees,you don't really have to worry about
the visual effects work of cleaning upyour tripod, or taking out the other
(45:35):
lenses on either side, or all that stuff.
And you can just start figuringout, like, where to put the camera.
Because that's the key.
Where do I put the camera?
How do I move the camera?
How do I block to this camera?
How do I tell stories with this camera?
You need to understand that before youdo anything on a huge scale, for sure.
And you can make really qualitycontent with a system like that.
(45:57):
And that system can be hadfor like under three grand.
And, or you can rent it on lensrentals for like a few hundred bucks.
And, um, get into that and juststart messing around with that.
I think Canon's tool to stitch thisstuff, uh, is like five bucks a month.
integrates directly with premiere.
Um, so there's, you know, there'sdefinitely, there's definitely
(46:19):
levels to it, but getting inand starting to develop your own
visual language is step number one.
Paul DeNigri (46:26):
Uh, what's some VR content that you're consuming right now?
That's got you excited.
What are you watching?
What are you playing?
Matt Celia (46:34):
Uh, great question.
Um, I spent a lot of time inheadset looking at the stuff that
we make at LightSail, so I feellike sometimes I don't get out to
see what, what else is out there.
Um, but two recent things that I reallyenjoyed is, um, on Apple vision pro
there's an experience called What If.
Uh, which was created byILM and, uh, and Marvel.
(46:54):
And it is a great example of storyliving because, you know, they
come into your living room andthey take you into these worlds and
you have some light interactivity.
Uh, and it made me think.
feel like I wanted to get back intothe Marvel universe and I wanted to,
like, kind of be a superhero again.
And like, like that was very, very cool.
And I thought that the teamthere did a really good job.
(47:16):
They just won an Emmy for it.
Um, I obviously have been watchinga lot of our Live in Red Rocks
series, uh, which is concerts.
Cause I think music works sowell in VR because you're, you're
on stage in a vantage point.
You can't even buy a ticket to.
And you get to see incredibleperformers making incredible music.
(47:37):
It sounds awesome.
You're at Red Rocks.
You're in this iconic venue.
So I really love that.
Um, and then let's see, whatelse have I been doing recently?
Uh, I play a lot of mini golf.
Walkabout mini golf is one of myfavorite things and easily the
thing I turn most people on towhen they're getting started in VR.
(47:57):
Um, and oh, so this week is Venice, uh,the Venice Film Festival and they have
an amazing immersive, like, island.
And so last week I had the opportunityto, uh, do a dress rehearsal, uh,
from a group called the FerrymanCollective and they have a new immersive
theater piece that's done in VRChat.
And that was incredible because Ilove theater and, uh, they've managed
(48:22):
to create these amazingly beautifulworlds that you get thrown into as an
avatar and get to interact with andfeel like you're part of the story.
Um, and it really takes that idea oflive story living to the next level.
And few things are, I think,as groundbreaking as the work
that they are doing, uh, in VR.
I think that they're justtelling some incredible stories.
(48:44):
Um, and it's really fun.
Paul DeNigris (48:47):
That's great.
Uh, what's up, what's upnext for you and LightSail?
What, uh, projects doyou guys have coming up?
Matt (48:54):
Um, so here at LightSail, you know, we're doing our Live at
Red Rocks concert series, uh, andwe've been busy filming season two
of that, which is very exciting.
So there's a lot of likeamazing acts with that.
We've been doing, um, someworkflows with Apple Vision Pro,
which we're pretty excited about.
It's all HDR.
So that's a whole notherwrinkle to things.
(49:14):
Um, uh, but you know, pushing thatquality level forwards, pushing the
resolution, pushing the color correction.
Um, and also building tools for alot of this stuff, because as we
encounter these problems in production,like how to live preview or how
to like even review visual effectscuts, we find that there isn't a...
a defined ecosystem of tools to helpreduce the friction for artists.
(49:39):
And so we've been heads down on acouple of tools that I, you know, I
hope are going to keep making thingsa little easier so that we can stay
being creative and less technical.
Paul DeNigris (49:50):
Yeah.
You guys have that app,uh, for the, the quest.
Screening room integrates with frame.io.
Matt Celia (49:58):
Yeah, exactly.
So we're working, we're working onrevamping that for, um, adding a couple
of much requested features for that.
Um, because it's just,we found it so helpful.
It's probably the app I'm in most onmy headset, to be honest, because I'm
constantly reviewing stuff and likemaking notes and watching down cuts.
And, uh, And it's really,really fast and easy to do that.
Paul DeNigris (50:21):
Yeah.
I remember, uh, you know, some, youwould review our shots through that
system and you must have a macro orsomething, uh, or something that you
can key quickly when you're in theheadset that just said VFX issue.
And then, and then you'd go back and putthe note, but I'd see them in real time.
And it'd be like, that's the VFX issue.
And I'm like, What's the issue?
(50:43):
I know.
Yeah, so we have, we have likehotkeys, um, just to do things quickly
when I don't have a keyboard, but theapp actually has like keyboard support.
Oh, nice.
So, um, when that's all like hookedup and doing that, I actually
just write the notes in real time.
Oh, that's great.
Um, which is, which is awesome.
But I love to add like annotationsupport so I could circle things
and, you know, anything that helpsartists like understand the issue
(51:05):
quicker is good because we just wantto, you You know, VR is challenging
enough as it is, so anything we cando to kind of make it easier, I think,
uh, helps make projects look better.
Paul De (51:16):
Yeah, that's fantastic.
Uh, anything else youwanna, you wanna plug?
Um, you guys got an Emmy nomination?
Matt Celia (51:24):
Yeah, um, we're, we're, we're stoked that our,
our show Live in Red Rocks VR isnominated for a Primetime Emmy.
Um, so I don't know when this airs, but,um, hopefully we'll, we'll have one,
but it's just an honor to be nominated.
There's so many amazing, um,creators in the category this year.
Uh, and I just think that it'sa good time for immersive video.
(51:44):
And I think that peopleare excited about it.
And I think that there's.
A lot of stories that work well at it.
Um, and we're just excited to keep pushingthat immersive ball forward at LightSail.
So it's a great, it's great fun.
Paul DeNi (51:57):
Yeah, that's awesome.
You guys have had a really, a reallygood couple of years between the
Emmy nom and, uh, just, just thegreat reviews on The Faceless Lady.
You know, calling out the, theexcellent, excellent technical work
and the storytelling, uh, work.
So yeah, you guys are, are doing greatstuff and, uh, um, tell us, tell the
viewers, you know, where they canfollow your, follow you on social
(52:20):
media or find your stuff on the web.
Matt Celia (52:22):
Yeah.
So, um, you know, always checkout our website, lightsailvr.Com.
And see most of our latest work.
If you want to follow mepersonally, I'm on Instagram.
Most days as a immersive_Matt.
Paul DeNigris (52:34):
All right.
Well, thanks so much, Matt, forsharing your, uh, your wealth
of experience in, uh, virtualreality and filmmaking and VFX.
It's like the whole package.
Uh, I'm sure that the, uh,my audience appreciates it.
And, uh, and I appreciate your time.
Matt Celia (52:50):
Cool.
It was fun.
Thanks for having me on.
It's always good to geek outabout VR and storytelling and,
you know, let's do it again.
Paul DeNigris (52:56):
Absolutely.
Thanks for Foxtrot X-Ray andthe VFX for Indies podcast.
I'm Paul DeNigris, thanksso much for watching.
Remember to like, and subscribe and checkout our back catalog at vfxforindies.com.
See you next time.
Step inside Kilok Castle, a magnificent stone fortress deep in
the Irish countryside and home to LadyMargaret, a tragic spirit doomed to
walk the crumbling manor for eternity.
Three couples compete to survive theghost's sadistic games in The Faceless
Lady, an immersive stereo VR experiencefrom Meta, Crypt TV, and Eli Roth.
This week on VFX for Indies.
(00:41):
Welcome to VFX for Indies, a podcastabout the intersection of visual
effects and independent film.
I'm your host, Paul DeNigris, visualeffects artist, filmmaker, and CEO
of boutique VFX shop, Foxtrot X-Ray.
Joining me on this episode todayis Matthew Celia, creative director
and managing partner at LightSailVR, A bleeding edge studio committed
(01:02):
to pushing the envelope in termsof storytelling in virtual reality.
Welcome to the podcast, Matt.
Matt Celia (01:07):
Hey, thanks for having me, Paul.
Good to be here.
Paul De (01:10):
So on this podcast, our goal is to educate filmmakers, usually
independent filmmakers who aren't supertechnical about aspects of visual effects.
And in this case, virtual reality.
So if you like what we're doing here,viewers, uh, if you like what we're
doing here, please like, and subscribe.
So you don't miss any of our newepisodes and you can check out our
back catalog on VFX for indies.
(01:33):
com.
My company Foxtrot X-Ray had theopportunity to collaborate with
Matt and the team at LightSail VR.
On The Faceless Lady, which Iteased at the top of the show.
We'll get to that in a second beforewe do Matt, just tell us a little
bit about yourself and your career.
It seems like you've been inthe VR space for a long time.
Matt Celia (01:49):
Yeah.
So thanks.
Um, I'm a, I'm a film school kid,you know, went to film school
back in 2003 Chapman University.
That's where I met my nowbusiness partner, Robert Watts.
Um, and you know, always was interestedin directing and creating content, but
always had a kind of a technical edge.
When I graduated, I went right intolike post production, working for Warner
(02:12):
Brothers, doing digital first content.
I ended up doing some commercials,working at a commercial
production company for a while.
And, uh, eventually landed into creatingmy own documentary, shopping that
around, and, uh, getting that hardreality of struggling to break into
the 2D, very competitive landscape.
(02:32):
Um, at that time, Robert, my businesspartner, he, handed me a piece of
cardboard with a phone inside it andsaid, Hey, I think this is like 2015.
He's like, Hey, I think thisis going to be big one day.
You should, you're technical.
You should take a look.
And you know, like creatively, I've alwayshad a artistic theater style to my stuff.
(02:52):
It's very real in the moment.
It doesn't really rely on a lot ofovert visual effects, I would say.
I mean, there's always likea lot of hidden stuff, but.
Yeah.
You know, I like to be in themoment and be in the real present.
And when I saw the piecewas called Walking New York.
Uh, it was by Chris Milk'scompany at the time.
And it was just like, I was in NewYork and I was in a helicopter and
(03:14):
it was just letting the scene playout and it felt like theater and it
really ignited a passion in the kindof storytelling that I wanted to do.
So I spent the next, like,48 hours watching furiously
everything I could find.
And I went down to VRLA, whichis a big conference here for VR.
And it was a bunch of hackedscience fair projects.
(03:37):
And everything I sawwas super interesting.
But what I noticed was everything Isaw was very much about technology.
And not about storytelling.
It was very much like, Hey, I puta camera down, isn't that cool?
But there wasn't that sense ofcharacter, that sense of conflict,
the sense of emotional stakesthat you get in a good story.
(04:00):
So I called Robert up and I said, Hey,I think as an artist, someone who's been
struggling to find how they fit into thisworld, I have something to say finally.
And I think this is it.
So we bought like 10 GoPros whenwe started making content and
people saw us on social media and.
We ended up creating one of the firstpieces for Paramount Pictures called, uh,
(04:21):
Paranormal Activity The Ghost Dimension.
It was a in universe original contentwritten by the writers of The Ghost
Dimension that, uh, tracks like thesethree kids summoning a ghost in a garage.
And it was one of the first VR piecesthat we were That had real people acting
scared had a mechanism that was, youknow, in universe, you know, we used all
(04:46):
the paranormal activity like blue and thefuzzy cameras fitzing out like really dove
into the world and the rules of the world.
And it resonated with folks.
Uh, got 10 million views.
Um, and we, we would walk around atconferences with like, uh, Gear VR in our
backpack and like hand it to people andlike they'd watch this horror content.
(05:06):
It was so wild.
Um, but they were like, no, that was cool.
I feel like I'm emotionally gripped by it.
And that was the thing for us that saidlike, Hey, we can take the a hundred
years of storytelling and put it intothis new medium and we can start to
figure out how to translate that.
And that is the like ethos of LightSailVR is we're story first people.
(05:28):
We're tech heads.
We're nerds.
Um, but at the end of the day, we workwith directors and creatives who want
to get into the virtual reality worldand experiment with this new medium.
And we translate how that actually works.
Um, and that's been whatwe've been doing since 2015.
So almost 10 years now,it's a pretty wild ride.
Paul DeNigris (05:51):
That's great.
Yeah.
I, you know, that was actually, my nextquestion was going to be about this
phrase story first, cause it's all overyour social media and your website.
Like it's, it's your mantra.
Obviously
Matt Celia (06:01):
it's, it's, it's a finding aspect of what we do.
And, you know, Robert and I.
We could do a lot of stuffand we've done a ton of stuff.
We've been very flexible as a company.
You know, we've done everything fromtraining to, to, uh, entertainment,
like The Faceless Lady, but ourheart is in entertainment and our
heart is in storytelling and theunique thing that I think LightSail
offers to a lot of productions isthat we speak both sides of that.
(06:25):
That chasm.
We are very technical, but wealso understand like the needs of
creating characters that emotionallygrip you and that puts you there.
And our phrase here is thatimmersive is not just about
the physical bounds of reality.
It's an emotional immersion as well.
And you can't forget that because youcan be immersed in a 2D film as much
(06:45):
as you can be immersed in a VR project.
It's about how good your story is.
Paul De (06:50):
So what are some of the things about VR that you think really
lend themselves to, to storytelling?
You know, what, what is it, whatis that secret sauce that you, that
you guys have discovered that makesstories compelling in the VR space?
Matt Celia (07:02):
Yeah, if I had to distill it down to one word, I
think that word would be presence.
It's the idea of feelinglike you're actually there.
Um, and if you start to think of all yourcreative decisions, As subservient to
that word, and that's your North Star,good things, I think, start to happen.
So, it's about feeling like you'restanding in the best spot in the
(07:26):
room, where you would naturally be.
A lot of early VR creators wouldput the camera in the dead middle
of the room, thinking like, Oh,that way I can see everything all
around me, because 360 at the time.
And if you actually go to a partyand you look at where people stand,
Nobody stands in the middle of a room.
Like, that just doesn't happen.
And so understanding, like, where a goodvantage point is helps feed that reality.
(07:50):
Like, how high a camera is.
That feeds that reality.
How people interact with the camera.
Feeds that reality.
And so, the idea is to make people feellike they are there because the greatest
advantage of being in VR Is that youlook around and you are in the story
and you want to give the audience thatagency, that choice to be able to choose.
(08:14):
I'm going to look at thepainting on the wall over here.
I'm going to look at, you know, thischaracter over here because that choice.
There's something, I don't know, there'ssome neurological thing about the way we
are built as human beings is where whenyou make that choice, when you initiate
that action, the connection you have tothe material is somehow stronger, and
(08:35):
this is backed up by dozens and dozensof studies in this field, and it's
why virtual reality is such a powerfultraining tool and a powerful, um, like
therapy tool, um, is because there issomething very real about you saying,
I looked at that, I looked at that.
That makes it a more active viewingexperience rather than when you watch
(08:58):
TV on your couch and you veg out, it'svery passive and things just fly by you.
That's okay too.
But VR, I think naturally asksyou to lean in a little bit.
Paul DeNigris (09:09):
Yeah.
I mean, it sounds like in a wayyou're giving a little bit of
directorial control to the viewer,kind of making them a partner, right?
You're, you're deciding this is thebest vantage point, but then the,
the viewer ultimately has the choiceof what they're looking at, right?
Yeah.
Matt Celia (09:23):
And, and, you know, VR has gone through,
um, VR is a big topic, right?
There's interactiveVR, there's passive VR.
When we started, we were doingeverything in 360 degrees.
And that was like.
People really had agency becauseyou could look behind you.
You could look anywhere and directingthat was a challenge because you
really had to think a lot about, allright, what are people going to do?
(09:47):
How are we going to get people to lookat the thing I want them to look at?
Now we've kind of evolved tomore of a 180 format for a lot of
narrative content, which I think is.
the best medium of this.
And you sacrifice a little bit ofthat immersion because obviously when
you look to your left, you see like ablack line that goes into nothingness.
(10:10):
But you know, as a creative, asa storyteller, I'm still saying,
Hey, generally look this way.
I'm generally framing my shots.
I can cut faster.
I know that you're going to belooking that way, but there's still
such a wide canvas that you stillhave that agency That it's the same
kind of agency you have when youwatch a concert or a theater show.
(10:34):
And you know, it's like that samekind of reason, like when you watch
a concert on television and they'recutting shot to shot to shot, it's cool.
That's one kind of the experience, butwould anyone say that that replaces
the value of going to a live concert?
I don't think so.
And I think it's because at a liveconcert, you're making that conscious
choice to look at the guitar player,look at the drummer, look at the singer.
(10:55):
Um, and it, it createsthat same kind of agency.
Paul DeNigris (11:00):
Uh, although most people are, most people
are looking at their phones.
That's true.
Which is a whole different thing.
That's one of the nice things aboutVR is it's hard to look at your phone
when you're, when you're consuming VR.
Matt Celia (11:12):
I mean, I think it's a great medium for, for creatives.
Yeah.
Like.
Think about it.
Right now, a lot of directors, theycreate content for phones or for,
for TV even, and they're watched inthe worst circumstances on earth.
Like, there's people talking, there'stexting, you're like making dinner, my
wife's folding laundry or doing whatever,like I'm trying to watch something.
(11:34):
When you're in VR, everything's gone.
The only thing that you arelooking at is the stuff that
the creator put in front of you.
And that is tremendously powerful, but atthe same time, I think that's a really big
responsibility because it does mean that.
If you can't keep people'sengagement like that, then you
(11:54):
have serious story problems.
Paul DeNigris (11:57):
Yeah, for sure.
And a lot of it, I think alsocomes down to the acting, right?
The acting has to be, has tobe engaging and naturalistic.
There's a, there's a moment in, uh,in The Faceless Lady that totally
sold me on the, on it as a show.
Uh, it's when, um, the, the, the,the lead character Ella, right.
(12:17):
It's when Ella is sitting on her bed.
In the room and she's she hasthis this emotional moment.
She's crying and we are we are sittinglike as if we are, you know, kneeling
on the floor in front of her likewe would if if we were trying to
comfort her in in that situation.
Right?
Yeah.
And it's like, okay, I'm I'm I'mtotally the In this world and on
(12:42):
this journey with these folks.
And then when, when some of the charactersdie and you, you guys put the camera
like right on the chest of the dude who'sdying and you watch him take his last
breath in VR and it's like, okay, that'sa, that's a real visceral experience.
Yes, I can look all over the place,but I don't want to because the acting
is so riveting and drawing me to it.
Matt Celia (13:04):
And I will say, like, John Ross, the director of
The Faceless Lady, was, is justa master of moving the camera.
And he really, we worked really closelytogether and prepped for that production.
We did a bunch of camera tests.
We did different heights.
We did different distances.
We did different angles.
And we put it all in a headset to tryto build the visual language of the
(13:26):
show, to know how far we could push,what would work, what would not work.
And a lot of those moves are theresult of many, many takes to get
it just perfectly dialed in, right?
To feel comfortable in VR.
Because there's this whole comfort feelingwhen you're moving the camera as well.
And I just think he did a reallygood job of Of listening and watching
(13:47):
references and thinking carefully aboutthe vantage points and the feelings
he wants to elicit out of audiences.
Like there are a number oftimes in the show where he does
something that does make you feela little like, Oh, I don't know.
Oh, that's a little weird.
You know?
And he used it not just because,Oh, this is a cool shot.
He was very specific about it.
(14:08):
He's like, no, I wantsomeone to feel off balance.
Their friend just died.
We should be feeling off balance.
And I love that he did that becauseI think that that is using the
medium to its fullest potential.
Paul DeNigris (14:20):
Yeah.
It's another dimension we don't havein, in 2d, uh, filmmaking at all.
So you've mentioned a couple oftimes, you've said the, you've said
360, you've said 180, uh, you know,for, for the, the folks out there
who maybe are not familiar with...
super well versed in VR.
Can you just talk like, you know,we've got 360, we've got 180, we've
(14:40):
got mono, we've got stereo, we'vegot passive, we've got interactive.
Could you just give us sort ofthe menu of, of the different,
uh, options out there?
Matt Celia (14:48):
So VR, like when you say VR, like think of VR,
like you think of television.
It is a very big and vast open medium.
Basically VR is anything consumedinside a virtual reality headset.
Um, so the kind of content that LightSailis most known for is passive, linear,
immersive video is kind of what I call it.
(15:09):
Um, and immersive video comesin a variety of flavors.
The first is 360, which is 360 degrees.
And that is usually captured with manycameras looking every single direction.
All around.
And so you can look up, down,left, right, backwards, forwards,
and you feel completely immersed.
It's like Google Street View,but like, you know, a video.
(15:32):
Um, those flavors also come in 2D,monoscopic, and 3D, stereoscopic formats.
Um, so, 180.
Is just the front half of that sphere,and typically 180 will most always be
stereoscopic because it's easier todo, and it really adds the immersion.
(15:56):
Um, but of course you can make itmonoscopic formats as well, and when
I say monoscopic and stereoscopic,that's like if you're watching, you
know, a 3D movie, because like you'rewearing a headset on your face, so you
might as well take the advantage thatyou have a lens of two different eyes.
And get that depth, because that depthcan be really exciting for storytelling.
And also, since you're working in aformat that doesn't have depth of field.
(16:20):
Right.
You don't have shallow depth offield and like a blurry background
playing with depth in a stereo senseis a really powerful storytelling
tool to enable you to say, thisis what I want to focus on.
You know, this is the core of my scene.
Paul DeNigris (16:37):
Gotcha.
So, so putting the most important thingsort of at that convergence point is a,
is a term of phrase that we used a lot.
Yeah.
Matt Celia (16:46):
Yeah, exactly.
And we don't, you know, a typical flat.
3D movie will move that convergencepoint to be what screen space is because
that's the most like the focal thingin VR It's slightly different because
we do it more like how you see whereasyour convergence point You're is way
in the distance because you don't seestereo depth past like 30 40 feet And
(17:10):
then everything comes out towards you.
So you only get that.
Um, I believe it's negativeparallax come towards you.
So you only really get that.
Um, but you know, it's about placingyour subjects in this like sweet
spot zone between like three and 15feet where things look particularly
good and sharp and accurate.
And that's like, kind of like in life.
(17:32):
When I walk up to you in the real world,I'm not standing 20 feet away from you.
I'm also not standing one foot from you.
I'm like, you know, thatthree to like six feet
Paul DeNigris (17:41):
realm, you know?
Right.
If you're six inches from the personyou're having a conversation with,
it gets a little uncomfortable.
Matt Celia (17:46):
It gets a little uncomfortable.
Paul DeNig (17:47):
Both in VR and in...
Matt Celia (17:49):
real life! And that goes back to, like,
creatively, thinking of the cameralike a person, and like, thinking about
where is your presence in the space.
Paul DeNi (17:59):
Uh, I imagine the use of sound is also really important to
help direct the viewer's attention.
Matt Celi (18:05):
I mean, less than you would think, to be quite honest.
Like, it's definitely a commonthing to say, like, use sound
to direct people's attention.
But the reality is, is that, um,unless you back it up with visual,
like, something visually that, like,Says, Oh, that is what I'm seeing.
There's so much going on.
There's so much sensory overloadthat it gets overlooked.
(18:27):
And like, that's a really importantpoint to bring up is that you have to
understand when someone put this headseton their face, especially they're new
to it, and they've got this enormouscanvas to look at, and they've got this
spatial sound and they've got the 3d,that is a lot of sensory information.
And it can be absolutelyoverwhelming to be in that.
(18:49):
Um, so you have to take it slowand you've kind of got to hit the
nail on the head a little bit.
Um, spatial sound is huge.
It's that thing that adds to presence.
And I would say it's almost more importantin VR than it is in a flat film, because
we always hear 360 degrees always.
(19:10):
And so having thatelement of it really does.
You know, tell your nervoussystem, Oh, I'm actually here.
Paul DeNigris (19:19):
Yeah, for sure.
Uh, particularly like the battle scenesat the beginning of The Faceless Lady.
Yeah.
You're hearing this, this full.
You know, world of these guysfighting and, you know, all of
the stuff, the fire and the swordsclanging and things like that.
And it, uh, and it really puts youin it in a way that, yeah, watching,
(19:46):
you know, with your soundbar underyour television, as good as that
may be, it's still not, you know,being beamed right into your ears.
Matt Celia (19:52):
And it's, um, it's head tracked, right.
I think people don't alwaysunderstand that it's actually,
it's It's real spatial.
So if I'm hearing a character talkit right in front of me and I turn
my head to the left, now I'm hearingit just in this ear, as if they are
still just right in front of me.
There's actually like a spatial level.
Anchoring of all the sound objects.
(20:15):
And that's the thing that reallyelevates it and makes that feel
like, Oh, I'm actually here.
Paul DeNig (20:20):
Uh, so you mentioned monoscopic and stereoscopic, so
one eye versus two eyes, right?
So, uh, just really quickly for, forthe non technical folks, the idea is
we see, we see two slightly differentimages, you know, one in each eye.
Normally, right?
If you've got, we've got stereoscopicvision, so we're seeing a little bit of
(20:41):
separation and that separation is whatour brain turns into three dimensions
and turns it to depth perception, right?
So stereo, we're taking advantage of that.
We've got a left eye imageand a right eye image.
Matt Celia (20:53):
Right.
And the cameras we use, the lenswe use specifically on The Faceless
Lady is the Canon dual fisheye, whichreally has a lens for this eye and a
lens for this eye, and that's what'scapturing onto a single sensor.
So you do get about 60 millimetersworth of, uh, parallax on that.
Yeah.
Which is, which is the averagefor, for, It's an average for
(21:13):
human, for human eyesight.
Everyone's between like 55 and 72.
Paul De (21:18):
As opposed to something like, uh, like avatar where they were
shooting with two complete camera rigsand this whole, you know, uh, complex
contraption to put the two lenses tohave two cameras and have one shooting
down into a mirror so that they cancreate the separation and all that.
And they can dial the separation.
Matt Celia (21:37):
You can shoot VR just like that.
But the problem becomes when yourfield of view is 180 degrees up.
Down left and right it, you would seethe other camera system right above
you, you know, and, and that becomesa huge rig and very complicated.
Paul DeNigris (21:52):
So, so that, that dual fisheye lens really has
kind of democratized the process.
Matt Celia (21:58):
It's been the biggest, uh, game changer.
I think when Canon showed up withthat lens and it's not just a
dual fisheye lens of like, it'sa really good dual fisheye lens.
Like it's an L series glass.
It's.
Amazingly sharp.
It's got very, very lowchromatic aberration.
It's got low flare.
(22:20):
It's got, um, very low distortion values.
So it's, it's It's a very highquality precision instrument.
And I think Canon did an incredible featof engineering with it, to be honest.
Paul (22:36):
Um, they certainly that's lens certainly makes the compositor's
job harder because we can't, we can't,you know, throw a couple of pixels
of blur on something to hide, hidethe seams, you know, when everything
in the frame is super sharp, um, youknow, everything that we added to
the frame had to be Equally sharp.
And there was no, a lot of the cheats thatwe do, uh, in, in regular 2d compositing,
(23:01):
um, yeah, VR is not available to us.
Matt Celia (23:04):
VR is just very unforgiving for visual effects.
You know, also you're takingwhat you do and you're blowing it
up like so large onto a screen.
So, and, but when you're working on it,you're working in it on a small computer
and it's like kind of tiny and it's,it's just, it's, it's the challenge.
Paul DeNigris (23:24):
Yeah.
So, you know, for, for those who arenot, not experienced with this, that
dual fish eye is going to shoot two.
Uh, two 4k images side by sideand they are spherical, right?
So they, they're not flatlines are not straight, right?
The, the, the photo, the pictureshanging on, on Matt's wall
behind him would all be curved.
(23:45):
All of those lines would be curved, right?
So now we have to, we're eitherworking in that spherical space and
making sure that everything we do isrespecting that curve, or we're taking.
A little section that we're going towork on and taking that curve away,
straightening it out, doing our work,and then putting the curve back on
and sticking it back in the frame.
So it adds a lot of complexity and alot of like these sort of geometric
(24:09):
gymnastics that we have to do to get toget everything to sit right in the world.
And then we're doing it in two eyes.
Matt Celia (24:16):
Right.
So what you do has to be exactlythe same on each eye, except
for your horizontal disparity.
Right?
And that's that's a huge challengebecause, you know, you're cutting the
character out right with some roto.
Well, you have to do the exact sameroto on the other eye, and then
you have to offset that, and thenmake sure that that's perfect, and
(24:38):
if you don't, then you're goingto have slightly different rotos.
So like, we did a project where avisual effects team tried to convince
us that they could use AI rotoscoping,and I said, it's not going to work.
And they were like, yeah, it'll work,and they went ahead and did it, and it
did not work, because the AI couldn'tdo it the exact same on each eye.
So sure.
The left eye looked fine and theright eye looked fine, but together
(25:02):
they were slightly differentand they did not look good.
Paul DeNigris (25:04):
Yeah.
Yeah.
We, we encountered a number of timeswhere, uh, the Roto was the issue, right?
Cause you've got, we'd have, we'dhave, um, Ella's running at one point
and she's running past the camera.
And so the parallax was drasticallydifferent from one eye to the next.
And the details of her hair weredifferent from one eye to the other.
(25:26):
And when we, the Roto, the Roto teamgave us mats that were perfect, right?
If you looked at left eye.
Perfect.
Look at the right eye.
They were perfect.
But then you put them together and it waslike, something's, something's not right.
Yeah.
Something's wrong here.
It doesn't feel right.
You know, that was a note thatyou guys gave us a lot was this
(25:47):
doesn't feel right stereoscopically.
And it sort of drove me crazy becauseI'm like, I'm looking at, I'm like, This
pixel lines up to this pixel and thenit lines up to this pixel in this eye,
Matt Celia (25:56):
but then you look at it and
there's like, there's that disparityand you're like, is it too far forward?
Is it too far back?
Like, and there's always this like littlebit of like grid warping juice that we
had to, like, we had to do, um, some,some roof replacements and like patches
took for ever because the 3d roof wasnot a hundred percent like aligned with
(26:18):
like the lens and we just had to like.
Sub pixel warp it, whichis challenging to do.
Uh, on huge footage files.
Paul DeNigris (26:28):
Um, and you guys, uh, you guys work in,
uh, resolve in fusion, correct?
We work in resolve, in fusion,um, primarily for financial reasons.
Um, just because it's, it offerspretty much the whole tool set we
need at a dramatically reduced price.
But, you know, on The FacelessLady, the thing that we explored
(26:50):
that was really powerful was theintegration with the actual timeline.
Cause we didn't have to do the visualeffects process of having to bake
plates out and export plates andcheck those and do all that stuff.
And artists could open up the timeline,select his or her clips, right
click, go create fusion composition.
And they had everything theyneeded in context right there.
(27:13):
Um, and that was, thatwas pretty powerful.
Um, saved us a little bit of time.
Yeah, that's gotta be really helpful.
How is, uh, fusion in terms of.
Being able to monitor yourwork in a headset in real time.
Matt Celia (27:25):
It's tough.
It does have support toplug a VR headset into it.
Um, but we've created a tool calledvr.ndi at LightSail, which uses NDI to
sling pixels and put it in a VR headset.
And so because we're a distributed shopand everybody dials in from wherever in
the world that they're working, all thecomputers in the closet have a plugin
(27:46):
called Nobe display and Nobe displaysends and eat NDI signal out of resolve.
And we use NDI bridge, which is freeby the way, to sling that back to the
artist's home computer, where theyhave a headset, uh, where that home
computer can receive that signal and thenbroadcast to their, their quest headset.
(28:06):
So they're able to seestuff it's compressed.
It's not always a hundred.
Like if you're doing the sub pixelwork, they downloaded it to their
local machine and did that sothey can have full res quality.
Um, but if you're just placingstereoscopic titles and you're just
like getting into the right space.
It's enough.
Paul DeNigr (28:25):
Yeah, that's great.
We, in my shop, we work in Nuke andNuke has the ability to feed a VR
headset, but it's not real time.
And you can't see your mouse andyou're in VR, you can't see your, you
know, your mouse cursor or anything.
So we, so it would be,you know, headset on.
Right.
And you set your perimeter.
So you could lean forward and you'dbe able to see through the world.
(28:48):
Yeah.
I'd be like.
Lean forward, tweak something,lean back, wait for it to refresh.
Matt Ce (28:53):
Yeah, or you do, or you do that thing where you take your
headset like on and off all the time.
You're like, uh, what is this?
What is this?
Um, I use the, um, I use the Quest Prowhen I'm at my desk and you can see
like it doesn't have the bottom bit.
And like, that's killer for me becauseI can see my keyboard and see my mouse.
Paul (29:09):
Oh, that's, that's great.
Yeah.
For me, the, uh, thechallenge was I wear glasses.
Yes.
So, so I would, that it was justkilling the bridge of my nose, taking
the head, the headset on and off.
And then I got the, uh, Zenni eyeweardoes the, uh, we're not sponsored by
Zenni, by the way, but Zenni, if you'relistening, I would love a sponsorship.
I love your glasses.
Uh, Zenni has the, uh, theinserts for the, for the quest.
(29:32):
Um, So I got those in my prescriptionand put those in and that was great.
But then I couldn't do the on off becauseI'm totally blind with no glasses.
So I'd have the headseton and could see clearly.
And then I take the, and I'mlooking at, you know, fuzz.
So
Mat (29:46):
I used to have glasses and, uh, it took working in VR for like seven
or eight years for me to finally go like.
I think I'm going to get the surgery.
I think I'm going to do theLASIK and, uh, it was worth it.
Paul DeNigris (30:00):
Yeah, smart move.
So in a typical show, um, and, and Iknow there's probably no such thing
as a typical show because every showis kind of, but, but typically how,
how is VFX fitting into your projects?
You know, how are you using the effects?
Matt Celia (30:19):
Here's the, the, the reality of VR filmmaking is every
single shot is a visual effect shot.
Every single one.
Because at the bare minimum, you're takingyour fisheye lenses and you're applying,
you know, an ST map or doing some sort ofunwrapping to get it into equirectangular
space and you're reconverging yourstereo and so on and so forth.
And then typically you'llhave stuff that needs cleanup.
(30:42):
Shadows, tripods, youknow, like gack, whatever.
Like there's, there's, uh, VR is a lotabout what's invisible visual effects.
And I think this has been talkedabout a lot, just in the broader
visual effects community.
Cause you have a lot ofproductions coming out saying like
we did it all practically.
There's no visual effects.
(31:03):
And you're like, whatare you talking about?
There's like 2, 000 visual effects shotsjust to clean up like, you know, just
to do a merge or a comp or take outa shadow or take out all this stuff.
And so.
Visual effects in VR is really aboutmaking the visible invisible, because
again, going back to that word presence,you want to maintain the idea of presence.
(31:28):
And if you're seeing a camera shadowright in front of you, because the
sun's behind you when you shot thescene, well, that's pretty distracting
because you're looking at a camerashadow instead of just the ground.
Um, and so there's a lot of that.
You know, I think we were reallyfortunate on The Faceless Lady to have the
opportunity to collaborate with you folksto do creative visual effects, because
(31:51):
that's where the real fun stuff happens.
But to be honest, because,you know, VR productions are
usually budgetarily challenged.
Um, sometimes you don't have, youknow, the coin to get even have fun.
Um, you're just, you know, spendingit all on taking out lights.
Paul DeNigris (32:10):
Yeah.
So, so VFX is.
Part and parcel of thewhole process, right?
It really is.
Okay.
So, so therefore VR creates, youknow, typically what I ask a, a
guest is what are some challenges andproblems that VFX helped you solve?
You, you just covered that, right?
Yeah.
I mean, effects is, is helping you guysall the time just to, just to make VR.
Matt Cel (32:33):
If you want to make VR before you go make anything in VR, the
key thing is you should take like abeginner intro visual effects class.
Just to understand thebasics of how stuff works.
You should understand,like, what a clean plate is.
You should understand howgreen screen actually works.
You should understandhow compositing works.
Like, you don't need to be anexpert artist and any of that stuff.
(32:55):
Like, lord knows I am not, like,the world's best roto artist,
or composite keyer, or whatever.
But I do understand the basics, sothat when I'm on set, I know, like,
hey, I need clean pixels there.
And if I don't get those cleanpixels, it's going to be a
really expensive cleanup job.
Because if you think about it, youhave 180 degrees field of view.
(33:16):
If you want cinematic lighting, you'regoing to need to put lights in your frame.
Like, or you're going to need to spend alot of money on very large sources, but
lighting with super large sources faraway from the talent means that you're
not going to get the look you want.
So you'll always paint out lights.
Well, now you want to move the camera.
(33:37):
Okay.
Well, that just became a lot harderbecause how do you capture a clean plate
and move the camera and do that paint?
That becomes really challenging.
Oh, you want your light over here?
Well, guess what?
Your whole camera shadow isnow on this opposite wall.
How are you going to clean that up?
And so, it is a frustrating,like, like, jigsaw puzzle.
(34:01):
Like, you just, you know what it is?
It's kind of like you're in the, you know,submarine and a pipe bursts and you put
your hand on it and then another thingbursts and you gotta go put your hand
on that and then like, and like, that isAll these things in visual effects is the
answer to solve all of these problems.
But if you, um, if you don't shootit correctly or think about it,
well, you know, every hand thatyou're covering those pipe is dollars
(34:24):
and your visual effects budgetcan quickly spiral out of control.
Paul DeNigris (34:28):
For sure.
And then on the flip side, havingto shoot for VFX creates a whole
other set of problems, right?
Because like you said, youhave to shoot clean plates.
You have to shoot greenscreens a particular way.
Uh, and sometimes it'splaying whack a mole.
It's this, this, thesame, the same metaphor.
I remember like, uh, we had thosedrone shots in the, in the hedge maze.
(34:52):
And we go back and forth and I'd fixone thing and you'd be like, here,
this is what I want you to fix.
And I'd fix that in the process offixing that I've knocked this other part
of the shot completely out of whack.
And you're like, well, that wasgood, like three versions ago.
Now it's all fucked up and,
Matt Ce (35:07):
and, and, and it's hard because as you're doing it, you
just version, version, version.
I mean, we go through stuff so oftenand I will say this too, in VR, because
there's such a huge frame of stuff.
Just QCing your own work ischallenging because you'll be
looking at this one part of the frameand be like, yeah, that's great.
(35:27):
That's awesome.
And you'll watch it four or fivetimes and you'll totally forget
about this part of the frame.
So, you know, if I were to distill thisconversation down to one sound bite
of advice is just give yourself moretime, more time to capture on set to
get the assets, more time in post toQC it, to watch it because every part
of the process has friction and there'sjust so much more that you're creating.
(35:51):
And you're working, you'reworking at 8K resolution.
It's huge.
You're working at 60 frames a second.
That's almost three X a normal film.
That's that's a lot of data.
And so, and then your frame is enormous.
So it just takes time.
It's not that it's so muchharder as much as it is.
(36:13):
It just takes more time.
Paul D (36:16):
Yeah, it takes more time to send the files to a, to a vendor,
to, you know, to download even withinyour internal team, you know, offsite
having to download stuff for, for,
Ma (36:26):
I mean, that's primarily why at LightSail, we centralized everything.
Paul DeNigris (36:30):
Yeah.
Matt Ce (36:30):
And we have people dial into our computers because we were
spending days just copying files to ahard drive and then shipping it off.
And then, oh, wait, therewas a problem with that.
Oh, well, I got to do that again.
You know, their drive and like.
You start to lose a lot of time.
Paul DeNig (36:45):
Yeah, I can imagine.
So you talked about, uh, John Ross,the director of a faceless lady
and his uh, facility with movingthe camera planning camera moves.
Can you talk about, you know, likewhat are, what are some of the,
the camera rigs that you guys used?
Are we, are we, are we steady cam?
Are we Dolly?
Are we on cranes?
Are we on drones?
You know, how are wegetting these cameras?
Matt Celia (37:07):
So, One of the big things that you have
to think about with VR is you needit to be as stable as possible.
Um, if you're, if you're having multipleaxes of rotation, it's a good way
to make your audience sick, right?
You want to let the, the audience bein control of pan, tilt, and roll.
(37:31):
You want to controlthings like forward dolly.
Lateral dolly, you know, you want tocontrol some of the bigger elements,
but you don't want your camera justdoing like a little bit of this
because they're going to feel liketheir whole world like doing this.
So I will say that, youknow, we stabilized over 100
shots in The Faceless Lady.
(37:52):
Using Syntheyes.
And it was a large portion of the workthat we did, because even when you're
on a Ronin, like a good old Ronin rigthat has like the, the skeleton arms
and is like a really good operator,um, there's still like a little bit
of that shake and a little bit of thatvibration and a little bit of the forward
and backward and six axis movements.
(38:13):
And that can be really, reallydisturbing in VR because you kind
of feel like you're doing this.
Even though your body is not actuallymoving and that separation is what creates
simulator sickness That's the one wordfor it, but really it's your like your
nervous system saying like something'snot right here So I should feel sick So
(38:34):
when we think about moving in camera,we typically stay on dollies because
they're rock solid dollies of courseare really challenging because You'll
see the track right underneath you.
So now again, you gotto paint out that track.
So what we'll do is we'll, we'llput the dolly on a long jib arm and
that'll be kind of what we rock.
(38:55):
So that way we just need more space,but we at least can pull backwards
and not see the edge of the track.
Um, and that's a really effective wayto move the camera around, to be honest.
Uh, cause we are able to getthese like nice and subtle moves.
We're able to control thespeed, we're able to control
acceleration and deceleration.
So it's very, very soft, very,very linear as much as we can.
(39:20):
And a lot of that is why wehave headset preview on set.
You know, John is in the headset, I'm inthe headset, while we're capturing this,
and this is why we did so many takes.
Is like, we just had to get it just right.
And we had an amazing crew in Ireland.
Like we had just an incredible gripteam, incredible camera operators.
Um, but it, it is unforgivingto move the camera.
(39:45):
Um, and John moved it a lot, whichwas great, but we wanted to make sure
that every movement is so motivatedabout being in that right spot because.
You have to remember, even whenyour camera's static, it'll still
feel like it's moving becauseyour audience is looking around.
So they're doing their pans,they're doing their tilts.
Paul DeN (40:07):
Yeah, it's really, the movement's really effective, especially in
the, in the hedge maze sequence, you know,because, because you're in, you're in that
space and you've got the hedge walls oneither side of you and the cameras pushing
down the, the, the, the row or, or pullingback and it's, and it's going by, did
you get, were those dolly shots as well?
Or was that...
Matt Celia (40:26):
so the hedge maze was not dolly shots.
The hedge maze was on a Ronin withlike these like skeletal arms and
a Rua running through the maze.
You know, trying to keepit as steady as possible.
We did a lot of post stabilizationon that stuff to clean it up and make
it look as comfortable as possible.
There was moments in that episode though,where it's definitely not comfortable.
(40:48):
And that's the conversation that Johnand I would go back and forth with a lot.
We'd say like, Hey, you know, we canonly use this much of it in the cut.
And then we need to sandwich itbetween two comfortable shots.
Cause we like to approach VR aslike having a comfort budget.
Whereas you can have one uncomfortableshot, but if you have like three
uncomfortable shots in a row, guess what?
(41:10):
Your user is just, they'returning off the headset.
They're getting out of it.
And you don't want that.
You want to be able to use it to saylike, I'm being chased by a ghost.
I should feel a little uncomfortable.
Not like I feel queasy and nauseousand need to go lie down for an hour.
Yeah.
That's a great point.
I mean, movement is one ofthose things that I think.
People are very, in VR,they're very scared to try.
(41:34):
And they're also toldthat they can't do it.
And the reality is, the truthis it's somewhere in between.
And I think John Ross figured that out.
I think he found that, like,threading of the needle, where
he created the sense of movement.
And you know, interestinglyenough, he actually looked at
Citizen Kane as a touchstone.
(41:55):
Um, because when you look at thatmovie, Those cameras were enormous.
You're not like popping those on a littlelike gimbal and flying around places.
They're huge cameras.
So all your movement has tobe careful and controlled and
orchestrated like almost a dance.
And then also that film surprisinglyhas a tremendously deep depth of field.
(42:17):
Like you'll see all the way out the windowand he's like, you know, On his sled
as they're, as they're talking in theforeground and they're moving the camera
and the characters are moving towards it.
And him and I looked at thatfilm a lot and we like broke down
certain sequences about like, thisis how the staging should feel.
And that became a real touchstone forThe Faceless Lady that I think worked.
Paul D (42:39):
Well, there you have it.
All for all of you.
Citizen Kane haters out there who,who complained that it's constantly
the number one greatest movie ofall time on everybody's lists.
It's pretty good.
It's a pretty damn good textbook,even even now, even in VR in
technology that that Orson Welles.
Couldn't possibly have dreamt of.
Matt Celia (42:59):
But has a lot of the same challenges.
Like VR is about taking what's oldand making it new again, you know?
Um, because it is thebirth of a new medium.
It's not going to replace TV movies oranything like that, but I think it's
fundamentally going to be a differentway for audiences to experience stories.
And that's what I think is reallyexciting about it is that we start to
walk this line a little bit betweenstorytelling and story living.
Paul DeNigris (43:23):
And that gets really exciting being that our
target audience for this podcastand our established listeners
and viewers are indie filmmakers.
They don't have the budgets ofMeta / Crypt TV / Eli Roth, but
they want to make something in VR.
They've got a story thatthat screams out for VR.
(43:45):
And you said, step number one, take somebasic VFX training, learn, learn that.
Okay, what's the next step?
What, how, how does, howdoes somebody jump into this?
Matt Celia (43:57):
Yeah, it's a great question.
I mean, the first question you have toask yourself if you're creating something
in VR is, is my story better in VR?
And that sounds like a, a dumbthing to say, but you'd be shocked.
Um, you have to have a good reasonto want to put people there.
Like, I think Robert Zemeckis has anew film coming out, I think called
Home, where it's like a single shotin a house through ages of time.
(44:22):
And I look at that and I go,that should have been in VR.
Because, like That just feelslike so right for the medium.
Um, but a lot of folks comeinto our door and they say, I
want to make Hardcore Henry.
I'm like, Hardcore Henry is not in VR.
Like that's a first POV.
It's all about like movingthe camera and crazy stuff.
The, the key to start out is toget a camera and start looking at
(44:47):
content, you know, getting intothe ecosystem of VR does not have
to be tremendously expensive.
Um, a really good, still pro levelquality can be found with like
the Canon EOS R7 with uh, the 3.
5 millimeter dual fisheye lens they have.
It doesn't do full 180degrees, does like 150 degrees.
(45:11):
Um, it's not a full framesensor, it's a crop sensor.
It's not 8K, it's 4K.
But, that means that the files area lot easier for you to work with.
Like, because working in4K is tremendously easier.
Working at 30 frames a secondis tremendously easier.
And because it's not a full 180 degrees,you don't really have to worry about
the visual effects work of cleaning upyour tripod, or taking out the other
(45:35):
lenses on either side, or all that stuff.
And you can just start figuringout, like, where to put the camera.
Because that's the key.
Where do I put the camera?
How do I move the camera?
How do I block to this camera?
How do I tell stories with this camera?
You need to understand that before youdo anything on a huge scale, for sure.
And you can make really qualitycontent with a system like that.
(45:57):
And that system can be hadfor like under three grand.
And, or you can rent it on lensrentals for like a few hundred bucks.
And, um, get into that and juststart messing around with that.
I think Canon's tool to stitch thisstuff, uh, is like five bucks a month.
integrates directly with premiere.
Um, so there's, you know, there'sdefinitely, there's definitely
(46:19):
levels to it, but getting inand starting to develop your own
visual language is step number one.
Paul DeNigri (46:26):
Uh, what's some VR content that you're consuming right now?
That's got you excited.
What are you watching?
What are you playing?
Matt Celia (46:34):
Uh, great question.
Um, I spent a lot of time inheadset looking at the stuff that
we make at LightSail, so I feellike sometimes I don't get out to
see what, what else is out there.
Um, but two recent things that I reallyenjoyed is, um, on Apple vision pro
there's an experience called What If.
Uh, which was created byILM and, uh, and Marvel.
(46:54):
And it is a great example of storyliving because, you know, they
come into your living room andthey take you into these worlds and
you have some light interactivity.
Uh, and it made me think.
feel like I wanted to get back intothe Marvel universe and I wanted to,
like, kind of be a superhero again.
And like, like that was very, very cool.
And I thought that the teamthere did a really good job.
(47:16):
They just won an Emmy for it.
Um, I obviously have been watchinga lot of our Live in Red Rocks
series, uh, which is concerts.
Cause I think music works sowell in VR because you're, you're
on stage in a vantage point.
You can't even buy a ticket to.
And you get to see incredibleperformers making incredible music.
(47:37):
It sounds awesome.
You're at Red Rocks.
You're in this iconic venue.
So I really love that.
Um, and then let's see, whatelse have I been doing recently?
Uh, I play a lot of mini golf.
Walkabout mini golf is one of myfavorite things and easily the
thing I turn most people on towhen they're getting started in VR.
(47:57):
Um, and oh, so this week is Venice, uh,the Venice Film Festival and they have
an amazing immersive, like, island.
And so last week I had the opportunityto, uh, do a dress rehearsal, uh,
from a group called the FerrymanCollective and they have a new immersive
theater piece that's done in VRChat.
And that was incredible because Ilove theater and, uh, they've managed
(48:22):
to create these amazingly beautifulworlds that you get thrown into as an
avatar and get to interact with andfeel like you're part of the story.
Um, and it really takes that idea oflive story living to the next level.
And few things are, I think,as groundbreaking as the work
that they are doing, uh, in VR.
I think that they're justtelling some incredible stories.
(48:44):
Um, and it's really fun.
Paul DeNigris (48:47):
That's great.
Uh, what's up, what's upnext for you and LightSail?
What, uh, projects doyou guys have coming up?
Matt (48:54):
Um, so here at LightSail, you know, we're doing our Live at
Red Rocks concert series, uh, andwe've been busy filming season two
of that, which is very exciting.
So there's a lot of likeamazing acts with that.
We've been doing, um, someworkflows with Apple Vision Pro,
which we're pretty excited about.
It's all HDR.
So that's a whole notherwrinkle to things.
(49:14):
Um, uh, but you know, pushing thatquality level forwards, pushing the
resolution, pushing the color correction.
Um, and also building tools for alot of this stuff, because as we
encounter these problems in production,like how to live preview or how
to like even review visual effectscuts, we find that there isn't a...
a defined ecosystem of tools to helpreduce the friction for artists.
(49:39):
And so we've been heads down on acouple of tools that I, you know, I
hope are going to keep making thingsa little easier so that we can stay
being creative and less technical.
Paul DeNigris (49:50):
Yeah.
You guys have that app,uh, for the, the quest.
Screening room integrates with frame.io.
Matt Celia (49:58):
Yeah, exactly.
So we're working, we're working onrevamping that for, um, adding a couple
of much requested features for that.
Um, because it's just,we found it so helpful.
It's probably the app I'm in most onmy headset, to be honest, because I'm
constantly reviewing stuff and likemaking notes and watching down cuts.
And, uh, And it's really,really fast and easy to do that.
Paul DeNigris (50:21):
Yeah.
I remember, uh, you know, some, youwould review our shots through that
system and you must have a macro orsomething, uh, or something that you
can key quickly when you're in theheadset that just said VFX issue.
And then, and then you'd go back and putthe note, but I'd see them in real time.
And it'd be like, that's the VFX issue.
And I'm like, What's the issue?
(50:43):
I know.
Yeah, so we have, we have likehotkeys, um, just to do things quickly
when I don't have a keyboard, but theapp actually has like keyboard support.
Oh, nice.
So, um, when that's all like hookedup and doing that, I actually
just write the notes in real time.
Oh, that's great.
Um, which is, which is awesome.
But I love to add like annotationsupport so I could circle things
and, you know, anything that helpsartists like understand the issue
(51:05):
quicker is good because we just wantto, you You know, VR is challenging
enough as it is, so anything we cando to kind of make it easier, I think,
uh, helps make projects look better.
Paul De (51:16):
Yeah, that's fantastic.
Uh, anything else youwanna, you wanna plug?
Um, you guys got an Emmy nomination?
Matt Celia (51:24):
Yeah, um, we're, we're, we're stoked that our,
our show Live in Red Rocks VR isnominated for a Primetime Emmy.
Um, so I don't know when this airs, but,um, hopefully we'll, we'll have one,
but it's just an honor to be nominated.
There's so many amazing, um,creators in the category this year.
Uh, and I just think that it'sa good time for immersive video.
(51:44):
And I think that peopleare excited about it.
And I think that there's.
A lot of stories that work well at it.
Um, and we're just excited to keep pushingthat immersive ball forward at LightSail.
So it's a great, it's great fun.
Paul DeNi (51:57):
Yeah, that's awesome.
You guys have had a really, a reallygood couple of years between the
Emmy nom and, uh, just, just thegreat reviews on The Faceless Lady.
You know, calling out the, theexcellent, excellent technical work
and the storytelling, uh, work.
So yeah, you guys are, are doing greatstuff and, uh, um, tell us, tell the
viewers, you know, where they canfollow your, follow you on social
(52:20):
media or find your stuff on the web.
Matt Celia (52:22):
Yeah.
So, um, you know, always checkout our website, lightsailvr.Com.
And see most of our latest work.
If you want to follow mepersonally, I'm on Instagram.
Most days as a immersive_Matt.
Paul DeNigris (52:34):
All right.
Well, thanks so much, Matt, forsharing your, uh, your wealth
of experience in, uh, virtualreality and filmmaking and VFX.
It's like the whole package.
Uh, I'm sure that the, uh,my audience appreciates it.
And, uh, and I appreciate your time.
Matt Celia (52:50):
Cool.
It was fun.
Thanks for having me on.
It's always good to geek outabout VR and storytelling and,
you know, let's do it again.
Paul DeNigris (52:56):
Absolutely.
Thanks for Foxtrot X-Ray andthe VFX for Indies podcast.
I'm Paul DeNigris, thanksso much for watching.
Remember to like, and subscribe and checkout our back catalog at vfxforindies.com.
See you next time.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Kingdom of Fraud
It’s the unlikeliest of criminal partnerships: a devout polygamist from an insular Utah sect joining forces with a shadowy Armenian tycoon from LA. The result - a billion dollar fraud conspiracy. In Kingdom of Fraud, investigative reporter Michele McPhee traces the origins of the extraordinary alliance between Jacob Kingston and Levon Termendzhyan. Together, the two men trigger the largest tax investigation in American history and weave around themselves a web of dirty cops, influential political relationships and transnational money laundering. All this is set against the backdrop of Jacob Kingston’s clan – The Order. A powerful and secretive polygamist organization in Salt Lake City. To whom Jacob is desperate to prove his worth. Kingdom of Fraud is produced by Novel for iHeart Podcasts. For more from Novel, visit https://novel.audio/. You can listen to new episodes of Kingdom of Fraud completely ad-free and 1 week early with an iHeart True Crime+ subscription, available exclusively on Apple Podcasts. Open your Apple Podcasts app, search for “iHeart True Crime+, and subscribe today!