Why This Podcast + Rob and PJ's Origin Stories

Enjoying the show? Hating the show? Want to let us know either way? Text us!

“What, another tech podcast? Why?!?”

Having been in the tech industry for multiple decades across a slew of different companies, Rob and PJ may not have seen it all…but they have seen a LOT.

More specifically, they have seen what works, what doesn’t and everything in between.

This podcast is a deep dive into the technical side, unafraid to take on the nitty gritty details. It also covers recent tech news with Rob and PJ’s honest (and sometimes brutal) take…

It is a podcast that isn’t afraid to ask “what the frigging hell is going on here?” Only they don’t say “frigging”.

In this inaugural episode, Rob and PJ talk about why they started this and dive into their origin stories for how they got where they are today…

Episode Transcript

(00:00):
Ierengaym.

(00:03):
com ierengaym.
com

pj_1_01-10-2024_153557 (00:11):
All right.
This is the inaugural episode ofTricky Bits with Robin pj.
In this episode, we're gonnadescribe who we are, how we got
here, and most importantly, whywe're doing this.
We've both been veterans of thetech industry for decades at

(00:32):
this point in time.
We've seen what works, we'veseen what doesn't.
We've seen what's fun and we'veseen what isn't.
And we're going to be yourguides on where tech is at,
where tech has been, and wherewe think tech is going.
So we're excited to have you onthe journey.
with us.

rob_1_01-10-2024_153557 (00:51):
what is the stated goal?
Where are we taking this?

pj_1_01-10-2024_153557 (00:55):
The goal of the podcast is we have seen a
lot of fun stuff in the techindustry.
We've been able to experienceit.
We've been able to build some ofit as well.
And I think there's a real joythat has been in the work that
we've done and.

(01:17):
There's also been a lot of soulsucking stuff we've experienced
in terms of how the business oftech, big tech, or the lack of
business in big tech has reallychanged the landscape.
So I think the goal that we'reaiming for here is really to try

(01:37):
to accentuate the deep,technical fun that exists for
the things that can be builtthat been built, and that we
think should be built as well ashighlight like where a lot of
the gaps are, where thesecompanies effectively shoot
themselves in the foot or damagethe industry because of really

(01:59):
ass backwards thinking.
And so I think ours is reallythis exploration between the
stuff that's super enjoyable andbeautiful and deep and technical
and fun, as well as the bullshitthat gets in our way.

rob_1_01-10-2024_153557 (02:13):
I would completely agree with that.
There is.
A lot of joy to be taken fromwatching people play games.
I think it's because we both inthe entertainment space for the,
the most part, likeentertainment, technology,
whether it be games or movies orspecial effects or whatever it,
may be, it's a consumer product.

(02:34):
The thing that we do isn't theconsumer product.
We just contribute to theconsumer product.
Um, there is a lot of joy to betaken from watching people enjoy
the final product.
And I completely agree that it'sbeen easy and it's been hard and
everything in between and overtime, that balance has changed.

pj_1_01-10-2024_153557 (02:55):
One of the most wonderful things, and I
think this is true both forgames and graphics in general,
is that we also had to solvewhat amounted to fun, deep
technical problems.
you're really getting at theguts of these things.
And asking these questions andnot just taking it for granted

(03:17):
where, oh, I, I won't even say Cversus Assembler at this point
in time, where you have folksthat are programming in Java or
Python or something that isbuilt on top of some kind of
virtual machine stack at somelevel, and you're not even
getting close to what the metalof that machine can do.
And I think it is a lot of funto dive that deep, especially in

(03:41):
the areas where there areproblems you're trying to solve,
make faster, make it moreefficient to push the boundary

rob_1_01-10-2024_153557 (03:48):
that to me is I think the essence of
tricky bits.
We're not gonna be scared ofgoing down to Assembler or even
transistors, in order to explainsomething.
I think we are gonna expect alot of our listeners, and
hopefully we can present thingsin a way that doesn't intimidate
people who doesn't understandit.
We can explain it in multipleways.

(04:10):
But I don't want to step awayfrom the details simply because
there's a handful of people whomight not understand it.

pj_1_01-10-2024_15 (04:18):
Absolutely., I'll fully admit, there was so
many things that I took forgranted, especially coming outta
school or even as a nascentprogrammer that really got
uncovered for me, especially atInsomniac games.
I.
When it really came time tofiguring out how are we gonna
make this as fast as possibleand as efficient as possible,
and as memory conservative aspossible and I found this life

(04:45):
changing frankly, in terms ofhow I started to look at all the
problems that came after that.
It influenced me greatly atDreamworks and beyond.
So I think this isextraordinarily valuable, that
people have a holistic view andcan reapply these lessons no
matter what level of the stackyou're at.

rob_1_01-10-2024_153557 (05:07):
Yeah, I agree.
I think a lot of it comes downto understanding the problem
that you are given.

pj_1_01-10-2024_153557 (05:13):
So Rob, how did you get into this whole
computer thingy in the firstplace?

Track 1 (05:18):
It's the only career that I have ever had.
I started when I was a kid.
I was fortunate enough to be a.
Born at the right time.
I guess I could've been bornearlier and it would've been
easier, but I was born in theseventies and I grew up in the
eighties.

(05:39):
And at that time in the uk,which is obviously where I'm
from, there was a great homecomputer scene.
We had all these new machinescoming out and people were all
trying to find their own way.
There was nothing right or wrongabout what people wanted to do.

(05:59):
And we had the, the Commodoremachines.
We had the, the Acorn machines.
The Spectrum machines, and thenthe UK had a whole bunch of
other random machines like theDragon and the Ttra and the oric
and a whole bunch of othercomputers who didn't go
anywhere.
My first computer, I always saymy first computer was A BBC.
Model B, which was a legendmachine, still is to this day.

(06:21):
And there's a whole story thereof how the BBC got involved and
wanted to do a, uh, a computerprogram for the nation and
wanted a machine that everyonecould have that was standard.
And they did.
It was, uh, quite successful.
Um, obviously the, the gem ofACORN was arm and the Archimedes
and the original arm processorcame out of that whole program.

(06:42):
And I must say thanks to myparents for getting that because
they were not cheap and theymust have sold a kidney or
something to be able to haveafforded it.
Maybe my dad was a drug dealer.
I got no idea.
But anyway, I got one and I veryquickly learned BBC Basic and
BBC Basic was brilliant andstill is.

(07:03):
And it had a built-in Assembler,which was the gateway to 65 0 2.
Like I said, everyone was doingtheir own thing, finding their
own path.
And I quickly found that mediaaudio graphics, which led down
the games and later the demopath was.
Where I wanted to do things.
And in that day, assembler wasthe only way to really get

(07:26):
there.
You had to move things quicklyon a slow machine.
You had to write an assembler,which is where my love of
Assembler originally came from.
I have lots of fond members ofmaking really cool things and my
first commercial softwareproducts, and I wrote audio
trackers and MIDI players and FMsynthesizers and software
renderers and all these thingscame later.

(07:48):
Some of them were on the EDUs,but always done in the mindset
of gotta do it as quick aspossible.
'cause if I can do it quick,then I can do more.
Or whoever I'm selling thissoftware to can do more without
me taking up the whole machine.
I went to college.
I never had a pc.
My first PC was after college.

(08:08):
I went to work for ChrysalisSoftware in all the Room
England, and they made all thegood ous games.
So IA was basically, they wereported from the Omega Bitmap
Brothers games, SIM City, 2000,all sorts of really cool games
for the Eds were all done byChrysalis.
So by this point I'd got myselfan Omega and I still think it's

(08:31):
the best machine ever made, for,for a programmers machine.
It was brilliant.
And because we were pouring outOmega games and some arcade
games to the Eds, I was inheaven.
It was like, this is awesome.
I get to write Assembler everyday and I get to ride these cool
games and it was awesome.

(08:52):
After Chrysalis, I got my firstPC while working at Chrysalis, I
didn't really program the PCmuch until I got to Dreamworks,
and this was a whole learningexperience for me.
I'd never done X 86.
I didn't barely knew what itwas.
I, I never used any of the PCapps.

(09:15):
I did all of my college stuff onthe eds.
They had the word processors andprinter drivers and laser
printers and blah, blah, blah.
I didn't need a pc and it wasmuch easier to program than a PC
was.
So when I got to Dreamworks,they're making trespasser and it
was in a very soy state.
It was very low performing, veryhigh memory, and obviously a pc.

(09:40):
And this was my first timeworking in a big team, making a
new ip.
I've done lots of ports, I'vedone lots of small teams.
Some of these ports you could doby yourself.
Some of them were two peopleworking on the same port at the
same time.
And this was now a big team.
And he did this and she didthat, and that's how it was.

(10:02):
And it was producers andlong-term schedules and things
like that.
And it was all new to me.
It was my first venture intowhat today you would call AAA
development.
And the game was actually prettygood at this moment in time.
It got worse before it gotreleased, and it should
ultimately been a run and gunshooter.

(10:23):
But that game had so much tech,it had.
The software renderer and it hadbump map in and per pixel
lighting.
And we then code curvature thebump maps to get smooth
surfaces.
And it had wavelet dynamic meshterrain, it had volumetric
clouds, it had inversekinematics.

(10:44):
and it was done in this giant cplus plus template, and then one
uber function that would justcall into all these templates
and.
That's where my hatred of c plusplus comes from because trying
to write that in assembler was anightmare.
And, but I did, I optimized itfor Pentium Tanium Pro, Pentium,
M-M-X-A-M-D showed up at somepoint.

(11:05):
I wanted a K six version, so Imade a K six with 3D now
version, the game was delayedbecause the gameplay was crap by
this point, and at the same sortof time, PCs were starting to
get hardware and 3D FX was out,blah, blah, blah.
And early games on using thehardware natively was starting

(11:28):
to come out.
And now these games were veryfast at higher resolution, but
didn't have the features we had.
So when we were told to make ahardware version.
So what do you do?
And we were kind of stuckbetween a rock and a hard place
and the solution was nevergreat.
It was never going to be great.
They should have made that gameinto a Toro run and gun style

(11:50):
game with some cool softwaretech and call it good, ship it.
Two years earlier, it would'vebeen great, but that's not what
happened.
But to this day, the tech inthat game is awesome.
And at the same time, Microsoftwas making a thing called
talisman.
And this is gonna predate mostpeople listening.

(12:11):
Uh, talisman was a similarsystem.
It was kind of a tile based reprojection system and but it
wasn't perspective.
Correct.
So it was crap.
And it ultimately got canned.
We did get the attention ofMicrosoft through doing this.
And ultimately that's how Iended up at Microsoft was

(12:33):
through this I worked on DirectXfor a bit and then ultimately
was on the founding members ofthe Xbox team.
All the system architecture forthe original Xbox.
And that was at that point wasjust lots of paper calculations.
It was like, okay, if, if Alphablending takes does these
operations on the bu, how fastcan we alpha blend?

(12:55):
How fast can we render?
How fast can we clear the depthbuffer, blah, blah, blah.
And if the CPU takes this muchbandwidth, assuming it's 90%
cash to access, how much is leftfor the GPU, blah blah blah.
And we went through this for awhole bunch of different GPUs,
different processes, and theexecutives, a typical Microsoft
decision picked gigapixel as thepeople to do it.

(13:19):
It was a tile based rendersimilar to power VR today, or
Apple or something like that.
And back then it was rubbish.
Today, I may think differentlyabout it, but back then it was
terrible.
We caused a huge stink andultimately got it replaced with
it Nvidia chip.
I think if we hadn't have donethat, Xbox would've gone the
same way as all the otherMicrosoft consumer products,

(13:41):
they've made that's supposed torevolutionize an industry and
never did.
And the reason is because ofdecisions like that where
executives will just go, oh no,we need to do this.
And it happened again on Xbox.
'cause we originally picked an AMD processor and Bill Gates
announced that the GDC withouttelling any of us that it's an
Intel processor.

(14:01):
And we didn't even know that ithad switched.
And Steve er switched it and hiscomment was, don't fuck up the
Intel account.
That was literally his words.
And we're like, okay.
So all that work we just did isnow mute and this X 86, this
compatible thing.
It's true to a limit, but at thelevel we were working at doing

(14:23):
like bus calculations, it's noteven close to true.
They're not even similar.
So we had to go back to all thatagain and do it again within N
Video as well.
So that was a kind of a mess,but very typical Microsoft back
then of like some externalperson would come in and be
like, well do this.
And they have say, and that'swhat happens.

(14:44):
So for a long time, Xbox was onvery, very fragile ground.
It al it was almost just a, a,uh, a bullet point on
Microsoft's list of failedprojects.
And fortunately that didn'thappen.
So we made the Xbox, we did thedemos, we made those big shiny
xs, which were kind of cool.
Ultimately I left Microsoft andfirst attempted starting my own

(15:07):
company.
This would've been about 2001,one two, uh, one.
I started, me, own Company.
My goal here was I could helppeople make Xbox games'cause I
know more about it than anybody'cause I built the damn thing
and it became fairly obviousthat actually a viable business.
But in doing this, I gotinvolved with the PlayStation
two quite a lot'cause everyonewas making PlayStation.

(15:28):
Two games with the Xbox Games.
And typically they were leadingon the PlayStation two and port
into Xbox.
And there was a lot of thingsthat were not similar.
In fact, nothing was similar.
On course there was twomachines.
So porting was kind of tricky.
I'd done lots of ports.
I got the hang of things andthat's what I was doing.
And I, I only did it for aboutthree or four months and
somehow, and I don't reallyremember how I met Ted Price,

(15:53):
and I don't know whether it wasby Fluke or whether it was
through the PSS two work I wasdoing.
I can't honestly remember how ithappened.
And it's like, well, we got aconversation going.
It ended up like, why didn'tcome work for us?
And it's like, yeah, why didn'tI come work for you?
So my dreams of doing my ownbusiness went on the back burner

(16:14):
and I went to Insomniac.
Uh, this was about 2001.
It's ratcheting Clan one, and Ididn't go to the Bahar office
because they were moving to thecurrent location in Burbank, the
old Lockheed building.
There I was working on the RatClank PSS two engine.
It was me and Al Hastings werebasically the only engine

(16:35):
programmers on that game We didRatchet one, ratchet two, and
Ratchet three.
And about this point, mark Cernycame to me and says, we're
starting this project called theICE Team.
It's the initiative for a CommonEngine.
And today it's just known as theSony Ice Team.
And, but that's its actual nameand it says it's gonna be a
Naughty Dog's offices.
They're in Santa Monica.

(16:56):
And I'd like it to be part ofit.
And it's, it's also like thisteam's gonna basically get early
access to the PlayStation threeand we'll start doing common
code, highly optimized code forthe PlayStation three for all
first parties.
So I, I ended up on this teamand we went to Japan a few times

(17:19):
and worked, we had secret accessto all of these really cool
simulators and emulators andlike cycle level accurate stuff.
And they had this GPU that itwas in-house.
It was called the Sony rs.
but it only did pixels.
It was envisioned that the SPUand the processor would do
vertices and they had this thingcalled the L-D-P-C-U, which to
this day, dunno how it works.

(17:39):
All the instructions were inJapanese.
Mark Cerny could read it, butAIM wasn't even quite sure how
this thing worked.
It was basically a asynchronousgatekeeper where you could let
the ES run out of order and itwould send them onto the pixel
shader in order, in submissionorder.
That is so it would rendercorrectly even if bits of it

(18:01):
were done outta order, whateverdidn't care.
Uh, it never actually existed.
It was just an idea as to how toconnect A to B.
A B in Es B, B and pixels.
So they made this Sony RS and itwas kind of power VR-ish.

(18:21):
In that it had no fixedpipeline.
Everything was in software.
The depth buffer, the alphablending it basically it made
you do everything, which in someways is great'cause you could do
anything you wanted to do.
So we spent a lot of timelooking at tools of how would
you program this thing?
And then we realized in doingthis, you had to schedule it
yourself too.

(18:42):
So we're like, why are wavesgoing through the GPU today?
And it automatically allocatesbased on resources that are of
valuable.
And you've got a shade that thatuses more memory, you might get
less threads going throughbecause you've run out of local
memory.
Likewise, if you use a lot ofregisters, you might not have
optimal number of threatsbecause you're gonna run out of
temporary storage.

(19:03):
You have to do all thatyourself, and you had to trigger
the passes yourself too.
So you'd like go to here, stop,go do to here again, then
continue.
It was this whole kind ofnightmare-ish scenario as to how
you had to write these shaders.
I wrote Bump M.
It took ages to write bump Mmapping optimal, no stalls at

(19:26):
all.
And it was all done on thissimulate, I believe, was called
olive, and it was a cycle levelsimulator of the os.
And it's like, this is gonna behard to program.
Like what tooling do we need?
What do we need?
A compiler, blah, blah blah.
And how does this integrate withLDPs a year?
With the vertices, we didn'teven consider that.
I was just like, give me threevertices and screen space and

(19:47):
I'll start there.
All that's a problem for adifferent day.
So it was hard to program andthen Sony made the thing, or at
least laid it out in silicon andrealized that this thing is not
possible to make.
It's huge, physically huge innumber of transistors.
And Sony weren't GPU architects.

(20:11):
I mean they, they were alreadybehind N Video and a MD at this
point of matrix and everyoneelse who existed in the early
two thousands.
So they did what the first thingyou were expected to do is you
start cutting things.
So they cut the things that tookthe most area and that was all
the good stuff.

(20:31):
So now it was even harder toprogram and less powerful and
ultimately useless.
'cause it didn't do vertices.
So there was a bit of a panic.
They started looking at, well,what if we put four SPS in?
Can we or four cells can werender in software?
So the ICE team were looking atlike, how would you render in
software with a 256 K buffer ineach SPU Yes, it was just two

(20:56):
business K sram, fast as hell,but that's all you got for
everything.
So that was code data, DMA,input, output, output buffers,
and all of that.
So we looked at software renderand you could definitely write a
software renderer and it wouldbe pretty decently quick.
And again, it fit me perfectlybecause I've wrote tons of them
by this point.
But now you're competing withhardware, so like feature sets

(21:19):
starts to become reallyimportant and shaders.
How do you deal with shader whenyou've got a software optimized
renderer?
How do you just drop in a shaderand make it optimal?
I mean, there has to be aboundary somewhere, and what
happens if this SPU does this 32by 32 block and this one does
this 32 by 32 block what you dowith the boundary and things
like that.
It was possible, but wouldn't begreat and it wouldn't compete

(21:39):
with the Xbox 360, which had areal GPU.
And that's ultimately the goalhere was outta the same time is
the 360 was coming out and ithad to compete.
So then they looked at like,well, let's put 128 pss, two
shaders, uh, shader cores in itand things like this.
And weird, esoteric things wherewe could have big blend units

(22:01):
that you could just feed backinto each other and do these
multi plus blenders, which iskind of how originally dx, now
the not DX 90, the NvidiaGForce, the original one, kind
of had these blend stages.
It didn't have shaders, it justhad these blenders that you
could program between each stageand you could kind of get it to
do various effects.
So no was the answer.

(22:25):
Yes, it's, it'll work, no, itwill won't compete with the 360,
It was a DX 10 class GPU and thePSS three only got a DX nine
class GPU in the end because inthe end they went to NVIDIA and
we product them.
I had some contacts on NVIDIAfrom doing the Xbox stuff and we
kind of pushed them Sony towardsthat direction.

(22:47):
It was very, no at the timebecause it's Sony and a, the
Japanese culture is not want togo out and ask for help,
especially from an Americancompany.
But it was the only way theywere ever gonna compete.
So they went to Nvidia, they gotthe RSX, which it was kind of an
odd ball.
It was never exact.

(23:07):
Chip was never used in any sortof graphics card, but obviously
it had vertices.
So now what do we do with thesps?
We don't need them.
They don't need to do verticesanymore.
Now they can do anything theywant.
So the ICE team going back tothat had this whole initiative
now to like, what can we makethese SPUs do?
And the answer is, if you do itfirst party code, you can make,

(23:29):
do anything you want.
If you are porting code like theUnreal engine, you are screwed
because everything was DMAbased.
You had the DMA in process dmmaout.
There was no I'll access thismemory.
You could access it and DMA itand write a template to hide it,
which I think is what Bungeedid.

(23:49):
They could just run code on theSPU and it would sign it with
DMA in the background and cashand things like that.
We being first party took theopposite approach where it was
just make it work, do whateveryou need to do.
It doesn't matter how ugly it's,and that was the saving grace of

(24:11):
the PS three.
It wasn't the Nvidia chip, itwas less powerful than the a MD
chip in the Xbox.
The saving grace indirectly wasNvidia because that freed up the
spu and it could have been anyGPU to be honest with you.
But, having those pus forphysics and audio and particles
and in the end we're likemanipulating frame buffers and

(24:35):
command buffers and all sorts ofreally questionably, dodgy
things on the spss.
Initially we didn't haveregister level access to the, to
the GPU.
And NVIDIA was very adamant thatwe would never get as they
always are, and they, there wasopen jail, that's all we got was
open GL and it's like, oh, thisis rubbish.
But they screwed up because theyprinted on the debug TTY, they

(24:59):
printed address of CommandBuffer is here.
So it's like, oh, okay, I'lljust point the debugger with
that and see what you put in it.
And so we'd called GL and we'dlook at the command buffer and
it's like, yep, that's actuallythe command buffer.
And it's like if I put my owndata in there, it'll execute
that too.
And we soon figured out likelots of the structure of the

(25:21):
command buffer.
I quickly realized it's the samekind of format.
As the Xbox was.
So all bits started to fittogether.
So we went on a whole projecthere of reverse engineering, the
RSX, and we made this thingcalled IG Render, which was
Insomnia Games render, and alsoICE Render two projects doing
the same thing.
And we'd just share as muchinformation as we could find.

(25:43):
We very quickly figured theentire GPU, including
instructions and instructionformats and everything, it took
quickly being a year or two.
It took a while and enough thatwe could use it directly without
having, uh, to use any ofOpenGL.
You had to call something like,if I remember, you'd call like

(26:04):
GL Flush and in one of theregisters when it returned to
you in one of the volatileregisters that the compiler
doesn't have to restore.
It left the pointer to thecontrol registers, so you could
call it and you have to pulllike R 12 or something like
that.
You'd call this function extractR 12 into a variable, and that

(26:24):
was the DMA controllers forputting gets, and you could now
start and stop the GPU yourself.
Ultimately all of the stuff wereverse engineered and made
inline in our own code,ultimately years later became
part of Lib GCM.
So the CPU overhead of oursystem was minimal and

(26:45):
resistance would not have beenpossible without that ultra low
overhead CPU code..
However, we submitted the gameand they're like, where's the
render code?
They're like, you can't do this.
You can't render yourself.
It's like, well, we are, andit's not possible without, so we
had a whole fight with'em.
I felt like it has to be thisway.
And in the end they're like,just make it all ICE render.
Get rid of IG render.

(27:05):
Make it all ice render.
So it's, it's kind of supportedand we'll allow it.
And the reason they won't allowit is they hadn't tested all the
register combinations.
Like if you set a texture, youset it A, B, C, D, E, you write
it, registered disorder.
But if you set AB and then lateronset C and D and then reset A
and then set E, they weren'tquite sure what the GP was gonna

(27:27):
do and it ultimately didn't doanything.
But they hadn't tested any ofthis, so that's why they won't
let us do it.
So they started to test some ofthe vector, uh, that.
Ice surrender would kick out.
And then we just used icesurrender.
It was the same thing.
It was all in line.
It was fine.
But anyway, that's, that was along time insomniac.

(27:48):
So after that I ultimately had akiddo.
Uh, this actually, I was born in2004 and it's probably 2008,
something like that.
Now I decided I was gonna leaveLA and move somewhere else.
I almost moved to Bend, Oregon,but ultimately moved to Boulder,
Colorado, which is where I amright now, and brought the kiddo

(28:11):
here.
'cause I didn't want to raiseher in la I wanted it to have
more of a childhood of the eyehad of outdoors north of
England, playing outside allthat.
So we moved to the foothills ofthe Rocky Mountains, just west
of Boulder.
And she grew up as a mountainkid and she loved her.
But I had to leave Insomniac.
So I came to Boulder and Ibecame a consultant for
Insomniac for many, many moreyears.

(28:33):
I worked on all the rest of thePS three games, the early PSS
four games, the Xbox game theymade.
I, uh, worked on that.
So basically the PSS four enginearchitecture was mine.
The PS five engine that theyhave now, as we've seen in the
leak, is kind of the same enginethey used to have.
So it's also mine.
So yeah, all those are mine.

(28:56):
But when I got to, uh, Boulderbecame a consultant, I actually
did finally start the companythat was gonna start 10 years
earlier.
So I became a consultant, uh,did a lot of work for Insomniac,
did a lot of work for Otoy aswell, which was early video
research at this point.
Still doing lots of path, stilldoing lots of.

(29:18):
Novel camera things.
They had, they still own lifestage.
So they, uh, at the early, earlylife stages, they had the first
functional one and the, thefirst geodesic one.
And I did a lot of work on that,A lot of data processing, a lot
of handling of the how do yousynchronize everything.
And now I guess it's just apackage you could buy and anyone

(29:42):
can have a life stage.
So the new ones do specular anddiffuse and all this.
If you know what life stage is,go look it up.
It's Paul Devic, ICT invented Soin doing that, I did a lot of
movie work, lots of specialeffects stuff, and it was always
like, oh, special effects inreal time would be great.
But you look at how a movie doesit and like, they're not shy of

(30:05):
modeling the whole scene.
If they have to, if they have toget, go and get an architect and
build this entire building as a3D model, precise, exact to what
it is in the real world, theywill.
They're not shy of, it needs tobe perfect.
It'll be perfect.
And if that moss on thatbuilding needs to be modeled, it
will be modeled.

(30:25):
And that was that.
So I was involved a lot withlike how shadows and how did you
put a real shadow on a fakeobject and how even more
difficult, how to put a fakeshadow on a real object.
And this is the, the art ofspecial effects is how it all
integrates together.

(30:45):
And So today it's more intuitivethan it was, but back then it
wasn't.
Anyway, all of this took me to,uh, MagicLeap where initially
ar, my view of initial AR wasthere's a lot of technical
problems to solve.
There's a lot of things thathave to be in place to make AR

(31:06):
or VR work at all, like headtracking and prediction and.
Abusing command buffers to getthat prediction date into the
GPU at the last possible second.
But once this is solved, it's acommodity product.
So I did a lot of that work.
It was all low level.
I always loved that work.
So I did a bunch of that.
And then a lot of it was how touse ar, and this still hasn't

(31:30):
been answered.
My view of ar, the ideal AR is amovie special effect.
It's indistinguishable from thereal world.
It's fake, but it integrateswith the real world perfectly.
Yeah, not gonna happen.
One day it might, I think today,AOL's in the like late 19, mid

(31:53):
1970s computer graphics, you canhave pixels, you can have jagged
lines.
It has to be pink apparently.
'cause everything in AR seems tobe pink.
So maybe it stands out.
I don't know.
Pink's everyone's favoritecolor.
It's actually really difficultand the type of AR matters to as
to whether it's mixed realityoverlay canvas stuff or whether

(32:14):
it's truly looking throughglass, putting pixels in the
world, magically hollow lensstyle.
And once you've solved thetechnical problems, you start to
solve the visual problems oflike, how do you do a shadow?
If you've got additive display,how do you render a shadow at
all?
You can't render black, addblack.

(32:35):
Nothing happens.
So.
How do you even put fake shadowon a fake object?
Like, I have a coffee table, Ihave an object, I want to cast a
shadow.
Technically not possible with arbecause you can't rent a black,
you can't rent anything darkerthan the scene.
You can't subtract the lightfrom what's coming in.
So you've gotta invent all thesenew techniques of like, how do

(32:57):
you do it?
And we, some crude ways of doingit was like, just render a gray
polygon over the whole screenand then you can subtract from
the gray to make it look likethere's a shadow.
But that reduces the contrastand dynamic range of the rest of
the scene.
And there's all issues withthat.
So, and then you get intolighting of like, okay, I have a
object that's in a room, and thelight turns on.

(33:17):
How quick does that object inthe, in your glass, have to
respond to the fact there's nowa wide light in it.
It was hard.
It was a blast and it was, noneof it worked great.
I mean, none of it's ready forconsumer space, and I think
that's true to this day.
Uh, this was 2015 or so and soalmost a decade ago.
It wasn't great.
The work was great, but theresults were impossibly

(33:39):
difficult and so demanding yourvision system of as a human is
so good at spotted defects.
It's not your vision system'snot great at all technically,
which is why AR works at all.
But it's incredibly good atspotting defects.
And if it's something it doesn'tlike, you'll just focus on that

(34:00):
and it's really difficult.
So after Magic Leap, after that.
I went to, uh, daiquiri did moreAR work and Daiquiri was not
meant to be.
They, they they're the ones whoconvinced me that AR is
practical and viable in a verycontrolled environment.
And enterprise industrial isthat controlled environment

(34:23):
where you could say these lightshave to stay on all the time.
That window has to stay closed.
Consumers don't live by therules.
So ar in the consumer space ismuch more difficult.
But Daco did indeed have anenterprise system where you
could wear glasses and it wouldshow you, take two of these, one
of those, put it here, do this,do that, and it would watch you
and tell you what to do.
And it was, kind of decent inits environment.

(34:46):
Something else Daiquiri wasworking at the time was
holographic displays.
Seamus was doing these and wehad these crazy elk cost devices
that would basically change therefractive index of a pixel.
And by changing the refractiveindex of a pixel, you can bend
light and if you can bend light,you can make holograms.
So we had these real timeholograms.

(35:08):
They were this big likeminiscule things, but you could
put lenses there and make itbigger.
It was a, yeah, so it was aprojected hologram from a one
plane and you've got a hologramin space and it worked great.
We made a whole API so you couldrender.
It's like a GL type api so youcould render things in
holographic space and it wascolored too.

(35:31):
That tech got spun off when thecompany went down that got spun
off into a company calledPacific Hologram, which Seamus
run, and I believe it stillexists.
Their website still exists.
It, uh, it's still in stealthmode according to the website,
but maybe I've just spilled thebeans.

(35:52):
Ultimately, I stopped doing thatand I actually missed a bit in
all this.
In all of this.
There was the ratchet HDcollection, which is before I
went to Magic Leap.
As I said, I started thiscompany.
I always wanted to start.
I worked for otoy.
We did all that in all of this.

(36:12):
Sony decided that they wanted tomake a HA PS three HD collection
of the Ratchet PSS two games.
They were looking at somecompanies like, oh, we'll just
port it to sea and do all this.
And it's like, I don't thinkyou, we can realize what you're
up against the game.
The engine's straight assembler.
There's no C it at all.

(36:33):
And it's all very DMA driven andvery asynchronous with all the
vector units.
I think it's Ted Price who saysthe only people who could do
this is either us or Rob,because he's now a consultant.
And he wrote the damn thing.
So somebody came and was like,yeah, make this game.
So I ended up making the HDcollection of the PS three with

(36:56):
a company called Idol Minds whowere in Broomfield, which is
next to Boulder.
And basically,'cause they hadtesters and DevNet access and
they were PS three developers.
So it was easy to do it withthem.
But yeah, it involved reverseengineering the entire game.
And there's a whole story thereas to like, we couldn't recover
any of the assets or the tools,so we did it all from the retail

(37:17):
disc.
So it was the best way to go inthe end, but kind of a nightmare
at the time.
So, but anyway, that was beforeMagic Leap, then Daiquiri, then
I still had this business thewhole time I was at Magic Leap
and at Daiquiri I still kept mycompany in the background, not
doing much.
And that stayed true.

(37:38):
Did a bunch of contract work formany different companies and
ultimately ended up with Apple.
And that's when I finally shutmy company down because Apple
being Apple is lots of rules andworked on the Vision Pro.
Yeah, indeed.
Worked on the Vision Pro.

(37:58):
And again, more ar, but thistime not real ar This is now
projected mixed reality as theycall it.
I call it projected ar'causeit's from a camera and you're
not looking through a piece of,you're not looking through
glass, which you project lightinto.
You're now looking at basicallya VR headset with cameras that.

(38:19):
Capture the real world render ontop of it and then present you
the whole image, which is verydifferent to real ar.
It's much easier to ha handlelatency in the mixed reality
Apple style projected AR than itis in the real world.
ar'cause real ar.
The real world's gonna do whatthe real world does and there's
nothing you can do to stop it.

(38:40):
Whereas with the camera basedar, you can subtract, you can
subtract light, you can doanything you want to do, but you
have all the display problemsand all things like that.
So I did that and I left Apple,oh, year, two years ago.
Oh, a year ago.
And vision programs are nextmonth.

(39:02):
So we'll see how it is.
It's pretty, from what I'veremember of it, it's a pretty
nice device and I still don'tthink it's consumer ready no
matter who makes it or what theymake.
The technical problems of livingin the real world are far bigger
than the technical problems ofmaking the device work.

(39:23):
As we found out at Magic Leap,as we found out at Daiquiri, I
don't think Apple have the staffall the inclination to solve
those problems.
Apple's not a special effectscompany.
They might solve the technicalproblems really nicely.
They might have displays, theymight have super low latency, it
might have great head cracking,but that only gives you ar.

(39:43):
It's what you do with it thenthat defines the product there
isn't any killer apps for ar andI think we should do a whole
article, a whole podcast on ARand go into the details of this
Uh, it's been a great ride.
Uh, I do a lot less of thisstuff now than I, than I used
to.
I think pass the button to a newgeneration is the same for real

(40:04):
world problems.
Like we've had our stab at it.
We've set up these greatframeworks, we've set up these
great platforms.
We've built understanding, westood on the shoulders of giants
when we started and.
Now it's time to pass it to thenext generation.
Let them have a go and maybethey'll do better.
Maybe they won't, but it's, it'stime The origin story I was

(40:28):
born, in the late seventies, soa lot of my early childhood
really was in the, reallythrough the eighties and early
nineties.
And one of the things that Iabsolutely loved was an
explosion of visual effects andvisual effects development that
was happening and it connectedso well with these stories.

(40:50):
It also resonated for me.
So I love the movie Tron.
It's one of my favorite movies.
And it hits all the buttons, Ireally love that it is this
wonderful combination ofengineering and math and art all
coming together at the sametime.
It happened at the same timethat we, my family had a
Commodore 64.
That was something that my, mydad got I think it was like 84 I

(41:14):
wanna say.
And what I started doing was, Ilearned basic, basically because
I would, load up games and, playthem and whatnot, toss'em in the
gist drive.
But then I was really curiousabout how to actually do this
stuff.
So my dad had a subscription toCommodore magazine I would read
the Commodore magazine as itwould come in and I would look

(41:37):
at all the code that was inthere for like little games or
little, like fireworks displaysor whatever, I was doing a lot
of that while we're still inFlorida.
And then after my parents gotdivorced, we ended up My mom and
my brothers and I ended upmoving initially back to New
York where I didn't have thecomputer for the longest time.
And then we ended up inProvidence, Rhode Island, early
high school, I ended up gettinga Commodore Omega 500, it was a

(42:01):
beautiful machine.
I, so I'll admittedly, I didn'tdive as far deep into that.
What I liked doing with it isactually there was a lot of
built-in tools.
You could just do a lot of verycool things with I was able to
get a copy of deluxe paint tooor something like that, which
you could do some reallyprimitive animation and 2D stuff
in.
And then I could actually evenoutput that to the VHS tape.

(42:24):
had a built in speechsynthesizer.
So I was able to use that formovies we were shooting at the
time in high school.
It was a lot of fun basically toeven just play with it as a
content production machine,which is what I was doing a lot
with it.
I think this was a wild Westtime for consumer computer
graphics.

(42:44):
'cause there was Callegari,there was like Lightworks I
think Autodesk, obviously hadthe high end stuff at the time.
3D Steel Max was happening.
Ari's True Space is actuallywhat I had.
And it was the computer teacherat my high school had either
gone to Brown or knew folks atBrown, and she was able to get a
copy of True Space.

(43:05):
So this was like true space twoor three.
And for me it was amazingbecause it was a full
environment.
It was basically like Maya plusa render.
Her and again, a very primitiveversion of Maya, but still it
had animation in it.
It had the set geometry whereyou could do intersection

(43:26):
addition and subtraction ofsolids, so you could do some
really cool stuff like with it.
And so I like, got it together awhole bunch of, I didn't have a
PC at the time in high school,but my friends did.
So I basically started bigborrowing and stealing time on
each of their machines orgetting them to do stuff so that
we could actually cobbletogether like all of these

(43:47):
different scenes and assets ofstarships fighting each other.
And then in order to get it offof, so I got it all rendered and
I ended up getting friendly withthe Browns Computer Graphics
Lab.
This is 96.
And I was able to like, able todo stuff there and render it
there.
And then they had, I think itwas Media Composer at the time

(44:09):
where I was able to actuallyoutput the scenes onto VHS.
cause every single time I said,okay, I gotta do this.
Like, how are we gonna get itoff?
I don't know yet.
We gotta figure out the nextproblem.
So it was always like, like wehad a little bit of stuff and
then we figured it out and thena little bit of stuff we figured
out from there.
And then we like, ultimately gotit done.
And that was exciting and fun,which was like, we didn't have

(44:31):
to necessarily know how to doeverything at the same time,
which we needed to do was reallyfigure out like how are we gonna
get it?
Just the next step that was okayand, figure it out and change
it.
So I ended up going to, toBoston University because I I
could I went there on because Icouldn't, we couldn't afford
really any other university.

(44:51):
I was like, so I was able to gothere on scholarship.
And the comical thing is I waslike excited to do computer
graphics type stuff, but I hadmisread the course catalog.
So the course catalog hadcomputer graphics in the
computer science departmentversus the computer engineering
department.
So I actually ended up going tothe computer engineering

(45:13):
department., Boston Universitywas unique in that it had a
computer science department thatcame out of its math department
and a computer engineeringdepartment that came out of its
engineering, like electricalengineering department.
So we actually had both of thesethings, which were similar but
tended to have different foci.
I ended up more, in, in manyways as a precursor to stuff I

(45:37):
would do in grad school, I endedup taking a lot of like signal
processing courses.
We had some basic courses thatwe had to take, so like just
understanding, convolution,understanding like digital
processing.
But then I took the graduatelevel courses as my,'cause you
needed to take a couple of thosein order to get your
undergraduate degree.
So I ended up taking like theDSB course for in undergrad,

(46:00):
which was hilarious because thatmeant I couldn't take it when I
actually stayed there for gradschool.
Now at the same time, basicallywas actually still shooting
movies.
I was doing some plays.
I picked up some improvisationalcomedy skills.
And then after I got a PC and Igot a board where I actually
could pull in the video, I wasstarting to do editing of those

(46:24):
videos.
In Adobe Premier 4.2, at the endof bu my undergrad, I didn't
really plot out stuff very well.
I tried to get into MIT's medialab.
didn't get into MIT.
So it was the second time theyrejected me.
And but what happened was theyended up getting a couple of
professors at BU that were doingcomputer vision in the computer

(46:47):
science department, and a lot ofimage and video processing in
the computer engineeringdepartment.
So I was able to swing it totake courses both ways.
Finally got my graphics coursethere.
And this is a, I'll tell atragic yet comical story at the
same time of why I neverproperly learned Rasterization
in a second.

(47:07):
But at the same time, during myfirst year of grad school, I
actually picked up a consultingjob working for a financial
company on Wall Street.
'cause my dad used to work thereand he had some people who
needed to do some, actuallyturned out to be some database
stuff, which I didn't knowanything about, but it was like,
all right, let's just learn thisstuff on the job, figure it out,

(47:29):
make some mistakes.
And, so I picked up a bit ofsequel, did a bit of Java.
I didn't know Pearl at the time.
It would've been really usefulfor doing the parsing I had to.
But I, so I did that for a yearwhile I was, my first year of
grad school, which I was alsodoing as a teaching fellow
teaching algorithms.
So I actually helped teach thealgorithms course and I was the

(47:51):
only person to volunteer forthat one.
'cause I was this found, I foundit interesting.
So I ended up spiel is inKellogg was in Jersey City.
My grandparents lived in Queens,New York.
I used to take the train fromQueens to underneath the World
Trade Center, the Twin Towers,and then hop another train to go

(48:12):
to New Jersey.
I stopped working there Augustof 2001.
And the.
the day that I was supposed tolearn rasterization, and they
still had the course.
That day was September 11th,2001.

(48:32):
So I missed that class probablyfor obvious reasons that day.
'cause it was just a crazy dayat that point in time.
And then never, I learned Raytracing fine, but never properly
learned like ization.
It was always a little bit of achip up my shoulder for that
one.
But I'll admit, like I was alittle bit listless in grad

(48:53):
school.
I couldn't quite figure it out.
I did squeak by with my master'sproject at the end, which was
trying, this goes back to mymovie thing.
My, my master's project wastrying to use stereo cameras to
improve lighting for greenscreen.
And doing stereo reconstructionat the time was a pain in the
ass.

(49:13):
It was basically doing it in ablock matching system with some
map estimation on top of it totry and constrain it a bit.
But it was some fun stuff whereI was able to take the two
stereo images, And because itwas a green screen, I was able
to take advantage of the alphavalue you can get out of it and
use phase correlation as aninitial, like seed for your map

(49:36):
estimation for when you're doingthe actual stereo stuff.
So it was like, there was acouple cool things there.
It wasn't in the endparticularly practical, but it
was, squeaked me by, and thenafter that I headed out to la so
I and a buddy took three weeksto drive out from Rhode Island
'cause I knew a lot of friendsand I couch surfed the entire

(49:58):
way.
And then I, in LA I didn'tactually have a job.
I crashed at a cousin's placefor a little while and worked a
lot of odd jobs from doing somemarketing assistantship to
technical consulting.
And then I ended up at I, Ianswered this was when I was
using Craigslist at the time toanswer for a position at uscs

(50:22):
Institute for CreativeTechnologies.
So ICT, which is where PaulDevic was at, he was doing like
light stage stuff there.
I was not working with Paul Iwas working offsite in this
little warehouse in Marina delRey called Flat World.
And their concept was,'causethey were working a lot with the

(50:43):
military.
Send a UAV through to collectcloud data points, reconstruct a
3D scene from that, and thenhave all these screens you could
put up so that people couldtrain through it really quickly.
So project the scene that you'rerendering onto these screens and
then, you can go through it.
And they had a ogre as theiropen source game engine.

(51:08):
And this was at the time wherethere was like a bazillion game
engines that were floatingabout.
Unreal was there at the time,but it wasn't great at that
moment of doing third personstuff, which is what we wanted.
could, ogre wasn't so I ended upreplacing, so and I didn't
really know much at the time,Rob, this is like really like
nascent stuff for me.

(51:28):
But I ended up like they hadused Dogar'cause it was free and
then they bought a license forGamb, like use Gamb.
And I was like, all right, letme figure out how to put this
thing So like I had to figureout how to like, just do all of
this stuff.
So I ended up rewriting thewhole system with Gabriel
clients.
'cause the ogre one, I can'tremember exactly what happened,

(51:50):
but it failed.
There was a day that came whereI had gotten everything up and
running, but we just hadn't donethe switch yet from the ogre one
to the Gabriel one.
And this day where they werelike having People come by and
see it.
We had to switch it.
So it was like, it was a littletrial by fire, but it worked.
And then I augmented with a fewother things I did.

(52:14):
I was, I did one of thosethings, like everyone writes
their own ray tracing engine atsome point in time.
At, especially in college, Imade the questionable decision
of writing my own scriptinglanguage for this thing.
And that was a very bad idea.
Later on, they replaced it withLu, which was a much better
idea.

(52:34):
Around about the same time,because we had seen a lot, some,
a bunch of success.
This is back in oh four, therewas this virtual soldier project
they wanted to do, where againthe concept for it at the time
was that there was significantcultural differences between,
let's say the US and other, likeat the time, we were in the,

(52:54):
like the US was knee-deep in theMiddle East for Gulf War ii.
And the idea was like therecould be certain things that are
considered aggressive byAmerican standards that were
actually just part of theculture.
So what they wanted to do is beable to have soldiers train with
kind of these sort of culturallyappropriate modules so that you

(53:17):
could get used to the culturalengagement at and do this at
scale to effectively preventlike bad things from happening,
like shooting people.
So what we did is we had thisspeech recognizer that we hooked
up to a primitive AI and then wetook a graphics client I wrote
with a virtual human that weproduced and we actually had a

(53:39):
couple different versions ofthis.
We proof of concept this once weactually used for a little while
Microsoft's, Microsoft had aspeech synthesizer in engine.
And what basically would do isit would kick out phone names.
And then we would map those tovises that we would then use to
animate the face in real time.
It didn't look great, but it wascool that we could do it on the

(54:01):
fly.
And then, we projected thisunder this piece of plastic that
if you lit it for properly, itlooked like a hologram.
So I did that for about like ayear and change.
And then I tried to get intoDreamworks animation'cause their
compositing group had a, anopening.
And I didn't get the job but itwas just after I'd interviewed

(54:23):
within Insomniac games.
and I had gotten an offer fromInsomniac, and when I asked
Dreamworks like, cause they saidI didn't have production
experience.
And it's always one of thosethings like how do you get
production experience if youdon't work at a production
company?
But I asked them like, would agame company count?
They said yes.
So I immediately said yes toinsomniac games which kicked my

(54:45):
ass.
Oh my God, man.
Like I, I'll be honest, like Idid pretty well in college and
then even my first couple ofjobs where I was high flying, I
thought, I was a hot shit and ohmy God, I did not know how much.
I didn't know.
Like I It was, it's likeembarrassing at this point in

(55:05):
time to look back and see like Ijust, I you all were, doing such
high tech stuff and like I.
I, I felt just like my knowledgeof was just gonna get totally
ripped down, which it was, whichwas great.
Honestly, it was one of thosethings that was like actually
highly necessary.

(55:26):
And I will always joke thoughfor when, for resistance Gavin
was the one who suggested ususing Ga Anarch.
And I said to him, I don't thinkthis is a good idea.
And then he overruled me andthen he left being the lead of
the tools team.
So just what the hell?
So I was left, that was leftdealing with this relationship

(55:47):
now at this point in time andmaking this work.
It was very much so I had to,and there was so much shit I
didn't know, and I feelembarrassed about it now, but
just as we got into the factthat it was, the memory
management was the biggestproblem.
and having to like, they, likeanarchic, rewrote it so we could
replace the memory manager.
I, was able to replace it in Lua'cause that was the scripting

(56:10):
language they were using.
And that's really where I likeencountered you the most.
Which I'll be honest, theconcept behind it was decent
because the idea was like, isthere a way so that designers.
Could like directly basicallyget their stuff in without
having to have a programmerprogram.
It, so like, could a designerbasically use Lua and the art

(56:35):
assets and the tool of an arc,export it and have it just work
inside the game?
Uh, yeah, I, and I, I, I dowanna comment that.
I mean, this was because I wasworking basically both in the
tools and on the sort ofrendering side.
It put me in contact with you.
And you intimidated the helloutta me.
Uh, like, um, like it was, itwas, uh, it was like, all right,

(57:00):
I don't wanna, I don't wanna bugRob, but I gotta bug Rob Like,
um, like they, they messed up.
Long jump here.
I I, I don't really know what todo.
This was like the, uh, one ofthe version

rob_1_01-11-2024_100319 (57:13):
I do remember fixing that.

Track 1 (57:15):
Yeah.
And I was just like, look likeLe needs this.
But I looked at the assembly andset jump and long jump were
identical and I was like, I've,I've, I've hit my sort of
knowledge and you gave me agreat patch that worked well.
So appreciate that.
And we were both tea drinkers.
I was, I've become more of acoffee drinker now, but you
introduced me to PG tips.

rob_1_01-11-2024_100319 (57:37):
I still drink PG tips.
I have

Track 1 (57:38):
still do.

rob_1_01-11-2024_100319 (57:39):
the morning.
I've got one in my hand rightnow'cause it's,

Track 1 (57:41):
Nice I remember the day you actually let headed out from
the office and you dropped offyour stash of tea with me.
And I was always touched bythat.
So anyway I was there for theend of resistance and then
comically, I was not gonna beput on ratchet.
I was gonna be put on RAresistance too.

(58:01):
And I actually was also talkingwith Sean McKay at the time
'cause he was looking for a newaudio guy.
and I was helping him puttogether like interview
questions for audio'cause of mysignal processing background.
And instead basically I, thatwas gonna be a possibility of
heading over to the audio side,but I actually, there was a
tools opening on the renderingteam, the lighting slash

(58:22):
rendering team at Dreamworks.
So I tried for the McGann andthen I got it that time.
So I ended up heading over toDreamworks Animation, which kind
of was, that dream I had ofworking in the movies.
So I spent the first year and ahalf as just like a programmer.
And I was on the lighting toolfor three weeks making

(58:45):
improvements.
And then after that they had athey were doing what was called
the mini farm, which wasbasically a distributed
renderer, interactive rendererbasically.
So they would take a fewmachines and they wanted to
split it up so they could havebetter iteration cycles before
doing the full blown render.
So I was the second put on that,and I was able to figure out a

(59:07):
bunch of stuff.
And then after that, they keptme doing different, like
rendering things like I workedon, like speeding up.
'cause the way the Dreamworksrender worked was two passes.
One which was a primaryvisibility cache.
And then the second, which wasthe lighting pass.

(59:27):
And it turned out that if that,that primary visibility cache
was nearly enough for both eyes.
So you could actually justrender a few more polygons for
both eyes.
And then.
Light both at the same time.
And if you didn't really careabout camera dependent stuff,
which a lot of time you didn't,depending on what layer you're

(59:49):
talking about, you couldactually just light it once and
you get some pretty impressiveimprovements in terms of
rendering times.
So anyway, I did that and then Igot tapped to become the tech
lead of rewriting the lightingtool because Dreamworks had hit
a scalability issue and a lot ofthe artists were actually not

(01:00:10):
doing lighting anymore.
They were doing like 80% oftheir job was file management
because the tools no longer,like they were built with this
notion that they could hold theentire context of the scene in
memory at the same time.
and as you started to get likefoliage and fur and giant like
crowds, like just was notpossible anymore.

(01:00:32):
So the idea was like we wantedto rewrite these tools from
scratch and also reinvent theworkflow, which was, it's funny
'cause when I went intoDreamworks I was like, man, this
code is nowhere near asefficient as like what Insomniac
was doing.
And I, I had again, a bit of anarrogant chip on my shoulder by
that point in time again.
But then having to rewrite toolsfor this magnitude and scale, I

(01:00:56):
was starting to get hit with alot of like humbling lessons at
that point in time.
And I got in my head and then wejust kept figuring out problem
after problem.
One of the things that happenedwas the sort of, I spent a year
and a half studying how thelighter worked, like figuring
out and talking with all them.

(01:01:17):
And I came up with a system thatI thought was really gonna be
awesome.
And they all agreed to it.
They thought it was great.
We implemented it was terrible,like they hated it.
So then that was one of thoseexistential moments of the
project.
And then some folks on theartist side changed.
Some folks on the engineeringside changed.
And ultimately we figured outbasically what they took, some

(01:01:39):
of the concepts that I had andkind of reworked it.
And one of the fun things herewas that when I looked at the
stuff they wanted to do Irealized a notable it would be
inefficient, but a version thatwas able to get up and running
was effectively to haul, go backto my old database days.

(01:02:00):
And I was able to effectively Icreated an API that was totally
clean, but underneath the hood Iwas able to use SQL light
effectively as a an engine.
Because so much of this stuffwas really just about managing,
like how Lights were attached toother things.
And so like underneath the hood,I had a, an implementation that

(01:02:20):
used this basically just to getthe job done.
The reason I mention this is'cause it's right around the
same time that we hired Markover and the API had none of
that, that that SQL stuff insideof it.
Mark came in, he was able torewrite the entire
implementation to be vastlyfaster and more scalable.

(01:02:41):
So I do wanna give credit wherecredit is due.
I was able to get it up andrunning, but it was always one
of these things where it waslike, at this point in time when
he did it, it was the thirditeration we had on this thing.
So it was definitely like eachtime we got like closer to hey,
it's on its feet now.
It's really functioning well nowit's super fast But it's funny

(01:03:03):
because it's right around thesame time as I was getting that
project done.
Dreamworks had been great, butit, the projects that were
available were starting to, todwindle and it was around the
same time that Google came,reached out to me.
And it was this was actuallyafter I had met Jen who's by
that point actually was my wife.

(01:03:23):
It's a whole separate storyafter we'd known each other for
six months.
And I had a, I had my EL on theway.
Google had reached out a fewtimes over LinkedIn.
I didn't, I kind of ignored themand then Jen was like, you
probably should just respond.
And so I was curious at the timeabout whether I could make it
through the Google interviewprocess just to see could, was

(01:03:46):
it possible?
So after that fourth time I didthe phone screen, I did the the
interviews, and then Iultimately got an offer uh, to
which I said no to three times.
'cause I still really lovedworking in movies.
What Google was offering mewasn't that great.
And then.
We sort of figured out there wasone possibility of going over to
the Niantic team which would'vebeen interesting.

(01:04:08):
Mark Doberman's group.
I'm not sure you remember ifyou've ever met him, or, yeah, I
figure you might.
But they were only doinginternal hires, so I headed over
to photos and it, it happenedactually around the same time as
my daughter's birth, so I did areset.
But I'll be honest and I spent alot of years over at Google in,
at first stint in threedifferent teams kind of feeling

(01:04:31):
out of place because that bigtech culture of you gotta do it
perfect the first time and writelong grad school-like documents
to actually do the designs.
And the stuff didn't reallyconnect with users, but we were
doing it because it was complex.
All of these things ran prettylike anathema to How I had come

(01:04:55):
up in the industry, which islike solving problems for
people, doing it multiple times,being okay with being wrong and,
pivoting where you needed to.
So I, I struggled quite a lot oftrying to figure out my place of
just what the hell's going onhere?
And it was like, okay, maybeit's me.
Maybe'cause every, like singletime I was like, maybe I'm not

(01:05:17):
learning the right stuff, or Ihaven't, I really realized that
there was a, like my perspectiveon the business was vastly
different from big techperspective on business, which
is that when I was in WallStreet or at Insomnia or at
Dreamworks, I got it like, and Igot what the business was,
especially at Insomnia andDreamworks, we sold content and

(01:05:39):
all the technology that we didwas in service of that.
And I, I realized that, thatGoogle had a whole slew of cost
centers and, but they weren'tactually connecting the dots to
the business model underneath.
I then tried to say okay, let mesee if I can propose some
projects to actually see if Ican we make photos, money?

(01:06:01):
And we tried to do it throughads.
It worked, it was someinteresting engagements, but.
Ultimately the product areawasn't that interested in it at
the time.
This is back in 2014 after mytwins got born.
So I tried a couple things.
They got shut down.
LA was not working out for meand my family at that point in
time.
So then someone said once, don'tyou check out Boulder.

(01:06:24):
And I was like, that's crazy.
I could never leave la.
And then I started looking atColorado and it was like, all
right, this seems interesting.
So we um, came out to Colorado.
I stepped foot in the firsttime, October 21st, 2015.
By December 17th, 2015, we hadclosed on our house.

(01:06:44):
So it was just like, bam.
Yeah.
And I had remembered that youwere out here.
And so I was like, oh, let mecontact Robin.
We, I, we, I, after I got here,we to chatting again.
I got to have a beer every oncein a while.
So I came over to Google, wentto payments, which was, again, a
sort of thing where it's a globespanning thing for Google.

(01:07:05):
I was like, we should make thisinto a, a, an actual product.
They didn't wanna do that.
Uh, So then I worked, went to adifferent ads research group and
they were doing some imagestuff, which was a last straw,
like gasp for me at that pointin time where I was just like, I
felt very much like a cog in themachine and not really doing
much of anything.

(01:07:26):
uh, someone I'd worked with atpayments to become the CTO of
Major League Baseball who hadcreated an office in Colorado.
So I was like, all right, let metry that.
And I head over there because Iwanted very much to do ar'cause
they've got all these greatscans of all the stadiums and
whatnot.
But it also happened to be thesame time when the Supreme

(01:07:46):
Court.
struck down the law againstsports gambling.
So then all of a sudden I becameeffectively the manager of a new
team which was to create agambling light app.
And I still wanted to try andkeep the AR thing alive, which
is when I tried to get youinvolved, but they struck that

(01:08:06):
one down, which was sad.
again, it's all about what kindof the winds of the business
have at that point in time, andsometimes it's just what's
trendy and hot.
So I did I learned a lot moreabout gambling than I ever
expected to.
So I went to Vegas a couple oftimes, which is not my place,
but had to go there to actuallyunderstand the legal side of

(01:08:27):
this.
And then around the same time,so my wife was pregnant with my
fourth child.
I, some of my friends at Googlesaid they were getting serious
about the business side ofthings, and I got I let myself
get fooled.
And I headed back over to Googleagain this time as an
engineering manager, where I hadan absurd number of reports

(01:08:49):
right before COVID hit.
So I learned a lot basicallyabout how to manage a team
remotely at that point in timeand get projects done.
Tried to basically like, so Itried again to try and pivot
into kind of more the businessside, but Google really wasn't
having that even then, which issuper bizarre, especially where
they're at now.

(01:09:11):
And uh, then I joined a riskteam for a little while, and
then finally I was like, letjust see if, get back into
media.
And I headed over to Twitchuntil yesterday when after
having reformulated a team lastyear, getting us all together,
deprecating a product, doinglike lots of stuff.
Twitch decided to have layoffs.

(01:09:33):
And I think one of the bigtakeaways as I was going through
all of this was much, much likewhere I, I really started from
this really strong place forwanting to do content graphics
computer animation games, andhaving it really be something
impactful to people.

(01:09:54):
I.
There's a lot of it where I lostmy way, like just by increments
and a lot of what I'm excitedabout now that I think that
there's actually a tremendousanalogy between that and what
happened with Big Tech, whichwas a lot of like really awesome

(01:10:14):
stuff that they legitimatelywent after and then kind of got
lost in the morass along theway.
So what I'm excited about now isbeing able to figure out better
pathways, not just to likegetting back to the content
side, the visual side for me,but where there are lessons more
generally that can help out theindustry, where the stuff that

(01:10:37):
we're doing here can actuallyhave positive impact so that
people aren't as lost.
People have a vision, peoplehave a goal that they're really
going after.
So that is like one of the metathings for me that I'm excited
about for all the stuff thatwe're doing.