November 16, 2025 • 43 mins
A taxi pulls up with no one in the front seat. Would you get in? We invited Waymo director Nick Pelly to take us from that first uncanny moment to the engineering that makes a driverless ride feel calm, confident and, by the data, far safer than most humans behind the wheel.
We walk through the full autonomy stack in plain English: how cameras, radar and LiDAR fuse into a single view of the world; how perception, prediction and planning work together to thread through double‑parked vans, nudge through gridlock and still behave like a good road citizen. Nick explains why Level 4 autonomy is about design domain as much as capability, why hardware still matters, and how redundancy handles blocked sensors, grime or failures without drama. We dig into machine learning at scale, from training on diverse city data to tens of billions of simulated miles, and how teams tune precision and recall so the car avoids both missed hazards and needless hard braking.
Beyond the ride, we zoom out to the business and the city. Phoenix offered a launchpad to build the marketplace, charging and fleet operations; San Francisco demanded handling a busy city and human‑like judgement; London beckons with dense streets and weather. We explore what happens as adoption grows: fewer parking lots, smoother traffic, motorway platoons, even intersections that need fewer lights when vehicles coordinate. Nick also shares his focus area—reliability and freeway fail‑safes—designing for worst‑case scenarios so the system exits danger gracefully at speed; this episode was recorded a week before Nick and the team announced highway driving!
If you’re curious about autonomous vehicles, safety, AI, urban mobility or just want to know what “robotaxi” really means, this conversation turns buzzwords into something you can picture—and maybe soon, ride. Enjoy the episode, then follow and share the show, and leave a quick review to help us bring you an even bigger season two.
Like, Subscribe, and Follow the Tech Overflow Podcast by visiting this link: https://linktr.ee/Techoverflowpodcast
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_00 (00:04):
Hello world and welcome to the Tech Overflow
Podcast.
I'm Hannah Clayton Langton.
SPEAKER_01 (00:09):
And I'm Hugh Williams, and we're the podcast
that explains technical conceptsto really smart people.
How are you, Hannah?
SPEAKER_00 (00:14):
I am great.
I'm on holiday.
I am a little bit disappointedto be back recording remotely,
but such is life.
What about you?
Is the jet lag still veryprevalent?
SPEAKER_01 (00:24):
Oh, the jet lag's super prevalent at uh Melbourne
to London, London to Melbournein a week thing uh really's
really turns you upside down.
We like the most jet set podcastever, because I think you're in
upstate New York.
SPEAKER_00 (00:33):
Is that right?
I am indeed in New Yorkrecording from my friend's guest
room.
So yeah, I don't go very manyplaces without my mic these
days.
SPEAKER_01 (00:41):
And then when this trip's over, you're done for the
year.
Are you gonna stay in London?
SPEAKER_00 (00:44):
Oh no, I think we're going to Paris at some point,
but that's just on the train.
So it doesn't really count, doesit?
SPEAKER_01 (00:52):
One of the cool things about Europe, I mean you
can just say stuff like that.
I'm popping over to Paris.
Uh that's not a concept forAustralians.
We don't pop anywhere.
SPEAKER_00 (00:58):
It's a good segue into our transport-based topic
for today, which is trulyaccidental, but I'm gonna sort
of stretch it to make it work.
So who've we got on today, Hugh?
SPEAKER_01 (01:07):
We have Nick Pelle, who is a director of engineering
at Waymo.
And for those of you who don'tknow Waymo, Waymo is uh a
company that's very well knownto people who live in Phoenix,
in Arizona, in the US, and tofolks who live in San Francisco
in California.
And it's a company that uhdevelops driverless cars, so
autonomous vehicles.
So vehicles you can actuallyjust hop into, they have no
(01:29):
driver, no human supervision,and they'll take you like a robo
taxi from A to B.
And Nick's somebody I knew uhfrom when I worked at Google
Hannah Awesome.
SPEAKER_00 (01:37):
Yeah, I love this phrase robo taxi, which isn't
something that I had heardbefore we started recording.
Um I learned a ton, and I thinkyou guys are going to enjoy it
too.
So we will get right into it.
SPEAKER_01 (01:46):
Yeah, awesome.
Let's get Nick on the show.
Nick Pelly, welcome to the TechOverflow podcast.
It's great to see you, myfriend.
Great to see you here.
Thanks for hosting me today.
SPEAKER_00 (01:56):
Yeah, we're really excited to have you here.
So today we are here to talkabout Waymo and autonomous
vehicles, which, by purecoincidence, since you agreeing
to do the podcast, Waymo haveannounced their London launch,
and we have a good deal of UKand London-based listeners.
So this is really exciting forus.
SPEAKER_02 (02:13):
Yeah, I'm I'm super excited to see Waymo expanding
to more cities and operating infive cities in the US.
And it's you know quite wellknown to those who are uh in
those cities and using theservice.
But to go international, to goto London, I've got family in
London too.
I'm really excited to bring thistechnology to more of the world.
SPEAKER_00 (02:34):
I think it's one of those things that it's really
hard to imagine until youexperience it.
So maybe we kick off with justlike bringing it to life.
Like, what's the customerexperience like of getting in a
Waymo?
Because I have to say, I've beenin an autonomous vehicle, which
we might talk about later, butI've not been in a Waymo.
SPEAKER_02 (02:50):
Yeah, there's there's definitely a variety of
vehicles with autonomousfeatures out there.
But let me describe what Waymois.
It's a robotaxi experience whereyou know you pull out your
phone, fail a vehicle, you know,much like you would do with
Uber.
In fact, in some places wepartner with Uber, and you know,
the vehicle comes to pick youup.
But in this case, the vehicle isempty.
(03:11):
It's gonna pull over on the sideof the road with no one inside.
You get a private vehicle, aprivate experience, and it's
going to fully autonomouslydrive you to your destination.
So this is a ride hailingservice.
It's what in the industry wedescribe as level four autonomy.
Uh, you know, if you'd likelater, we can get into some of
those different levels.
The level four meaning, youknow, there is no human in the
(03:33):
vehicle that is ready to takecontrol of the steering wheel.
There's no one in the vehiclewho needs a driver's license.
SPEAKER_01 (03:39):
I remember when I was at Google, they had those
little uh Waymo bubble cars.
It looks like things haveevolved a lot since then.
So, what's the actual hardwarethese days?
What sort of what sort of a carare we hopping into?
SPEAKER_02 (03:49):
The uh vehicles that we have today, the base vehicle
is a is a Jaguar, Jaguar iPace,but it's been significantly
retrofitted by us to add all ofthe sensors that we need for
autonomous driving.
Cameras, radars, LIDAR, inertialmotion units, IMUs, so that it
can see the world very clearly.
(04:12):
We will be introducing othervehicles, uh, you know, several
that are in testing at thisstage that we hope to bring you
know to the public soon.
SPEAKER_00 (04:19):
And if I'm to put my product management hat on for a
minute, can we start with likethe why?
So, what is the promise of thesedriverless cars?
Why do we need them other thanbeing able to play music and not
having to speak to the Uberdriver when we're going from A
to B?
SPEAKER_02 (04:33):
That's the most noticeable uh for the user is
you get a private experience,and that's a really big win.
But what we think a lot about isthe 40,000 road deaths just in
the US, 40,000 road deaths peryear that are completely
avoidable.
And the safety benefits that anautonomous vehicle bring are
quite dramatic.
(04:54):
We've now driven over a hundredmillion fully autonomous miles.
And you know, we've looked backat that data, and we have
between five and 10x lessaccidents than human drivers who
uh are driving in the samegeographies.
The safety benefit is is reallyquite stark, as well as the
convenience that you mentioned.
SPEAKER_00 (05:15):
That's really interesting because whenever I
mention autonomous vehicles toanyone I know, whether it's
friends or family, admittedly,no one's really in the industry,
but their first sort ofhesitation always comes down to
safety.
So maybe later in the episode wecan get more into like why it's
actually more safe and provingitself to be, because it's very
(05:36):
counterintuitive, at least ifyou don't understand the tech
behind it and why it might besuperior to human drivers.
SPEAKER_02 (05:41):
Yeah, I think that's many people's reactions.
But what we observe is evenafter the first ride, people can
appreciate the safety very, veryquickly because within a few
minutes of driving, you will seehow confident the vehicle is and
it's seeing things, it doesn'tget distracted, right?
It's looking 360 degrees all thetime.
And folks report feeling thatconfidence even within the first
(06:05):
few minutes of riding.
SPEAKER_00 (06:06):
Interesting.
So I think I mentioned justbefore that I've not been in a
Waymo, but if we can cast ourminds back to like 2015, so I
went in an autonomous vehicle inAbu Dhabi in this place called
Mazdar City.
And I remember it having quitesignificant expectations of what
a driverless vehicle would belike because my parents were
living out there and they said,Oh, you know, when you come and
(06:28):
visit, we'll take you in thesesuper cool autonomous vehicles.
And when I went and actuallyrode in one, it was kind of more
like a little shuttle ratherthan like a vehicle that
actually was a car without adriver.
So is that like the earlyiteration of the technology?
Was this sort of shuttleservice?
And now has it evolved pastthat?
SPEAKER_02 (06:46):
Yeah, maybe let me lay out the capabilities a
little bit.
Um, I think this shuttleservice, did it have a driver
inside?
SPEAKER_00 (06:53):
There was no driver, no.
SPEAKER_02 (06:54):
Okay.
So the SAE have levels ofautonomy for vehicles from
what's the SAE?
SPEAKER_00 (07:00):
Sorry, Nick.
SPEAKER_02 (07:01):
The Society of Automotive Engineers.
SPEAKER_00 (07:03):
Okay.
SPEAKER_02 (07:04):
So what you experienced there was a level
four uh autonomous vehicle, andthat's the same as what Waymo
produces level four.
So level one is a is a manuallydriven vehicle.
Level two are vehicles that havesome autonomous features that
require constant humansupervision.
And this is actually very commonin any vehicle that you buy
today.
This is things like cruisecontrol, lane keep assist.
(07:26):
You know, they can significantlyreduce the load on the driver
because many tasks are beingassisted by the vehicle, but the
driver has to fully superviseand be prepared to take over at
any time.
Um then you have level three.
SPEAKER_01 (07:40):
Nick, sorry, with with level two, so you know,
you've got things like adaptivecruise control and the you know,
the lane guidance and things.
Does does Tesla's fullself-driving mode kind of fit
into level two?
So really you've got a humanthere who's supervising whatever
the car's capable of?
SPEAKER_02 (07:54):
Yep, exactly.
I'd say Tesla's FSD is a is avery capable level two.
It actually provides prettysignificant assistance, but but
it's still in the level twocategory.
And that's where there's anotherdimension that I was going to
get to in a moment.
You know, you've got the levelsof autonomy, which is really
about how much supervision isneeded.
And then you've got, you know,what is the operational design
(08:15):
domain?
The the ODD is the terminologywe use for those autonomy
features.
Just how much are they capableof?
So the shuttle that you werereferencing, Hannah, in Abu
Dhabi was level four, but with avery restricted operational
design domain.
It's probably just doingpoint-to-point service.
I imagine this is an environmentthat is not dense with people,
(08:37):
with other traffic, and it'sprobably limited to a modest top
speed.
SPEAKER_00 (08:42):
Exactly.
So if I remember correctly,Mazdar City was almost like a
campus, and you actually parkedyour car like outside and went
in.
The autonomous vehicle didn'teven look or feel like a car
when you rode in it.
It was kind of like this littlepod.
Yep.
And there were no other cars onthe road.
There might have been a fewpeople walking around.
I actually can't rememberbecause it was a while ago.
(09:02):
But it it wasn't replicating thetypical driving experience,
which is interesting becauseit's actually the same level as
Waymo, as you've just sort ofbroken out.
Yeah.
SPEAKER_02 (09:10):
That's right.
It's level four, but but with amore constrained ODD.
And this kind of gets to beginsto get to the heart of you know
what is so odd about building anautonomous system.
You know, you can get to levelfour on a constrained ODD by,
you know, just being verycautious, you know, limiting the
(09:30):
top speed.
And I'd imagine that vehiclewould react pretty quickly to
any object in its presence,right?
If it sees anything, it could bea pedestrian, it would just come
to a stop and not make forwardprogress until it was quite sure
the path is clear.
And that works okay inenvironments that aren't that
dense, that are relativelystructured.
That's not going to work if youput it into the middle of a busy
(09:52):
city like San Francisco orLondon, because it's not going
to make forward progress.
There's going to be too muchactivity, too much going on.
It can't reason about thisenvironment in a sophisticated
enough way to both be safe andto make forward progress in the
environment.
SPEAKER_00 (10:07):
Okay.
So in order to make forwardprogress safely, the
self-driving system, I guess,has to replicate what a driver
would do, or it sounds likemaybe even perform those actions
better, which involves likeperceiving what's around it and
making decisions as a result ofthat.
So how does that all happen?
You mentioned LiDAR.
Um, I'm sure there's a bunch ofother stuff.
(10:27):
Could you talk at a high level,talk the listeners through how
that all hangs together?
SPEAKER_02 (10:31):
Yeah, let me give a high-level overview of a typical
autonomous vehicle um drivingstack, right?
And then we can come back andkind of dig into why this is so
hard to be both safe and for thevehicle to you know make
progress in complicatedsituations.
So at the high level, you've gotall the sensors mentioned
earlier, cameras, radars,lidars, um, audio.
(10:51):
We also have microphones.
SPEAKER_00 (10:53):
Can I ask you a quick question?
Sorry to jump in.
The difference between a radarand a lidar, could you just talk
the listeners through that?
SPEAKER_02 (10:58):
Radars are working on sending out radio frequency
pulses and you know receivingthe returns.
And they are fantastic forlooking for objects that are
reflective to the RF frequency,which is typically metallic
objects, and they give you avery good speed indication.
You get this Doppler return.
That's why police use radarspeed guns.
(11:20):
LIDARs are actually sending outlight pulses instead.
LIDARs are the rotating objectsthat you see on top of the Waymo
vehicles.
You know, they really stand outvisually.
They're very sophisticatedpieces of equipment that are
sending out laser pulses andthen receiving the returns.
And they give very accuratemeasurement of distance to an
object.
SPEAKER_00 (11:40):
So is there like an interplay there between the
lidar knowing the distance andthe radar knowing the speed?
And then like somewhere in thesystem, a calculation of how
likely a collision is.
SPEAKER_02 (11:50):
Yes, and a sort of simplified version of how this
works, and is maybe how thingsworked 10 years ago, is you
would use your lidars to figureout that there is some object in
front of you.
You would then go to thecameras, which are very good,
they're very high resolution,and they can help identify what
the object is.
And then the radar provides theadditional, okay, how fast is
(12:13):
that object moving?
Now that's a very simplifiedversion of how things work.
What happens in the more modernsystems is that all these data
points are fused together.
So you could imagine you haveall of this rich information
coming from lidars, radars,cameras, and very early in the
pipeline.
So before you've even determinedthat there is an object, you're
(12:34):
actually fusing all theinformation together.
The strengths of radar, thestrength of LIDAR, the strength
of camera.
Modern AI allows you to reallyblend those very early in the
pipeline.
And that's at the perceptionphase, which is the next stage
of the pipeline after thesensors.
Perception is consuming, youknow, this is this very high
bandwidth data from the from thecameras, radars, lidars, and
(12:56):
forming a view, like a semanticview of the world.
Okay.
What does the roadway look like?
What are the vehicles?
What are their speeds?
What are their directions oftravel?
Pedestrians, other agents, youknow, cyclists, uh, people on
scooters, people on bicycles,static objects.
So it's it's building a semanticrepresentation of the world.
(13:18):
From perception, we then go toprediction because you need to
then play that representationforwards.
Okay, how do we think thevehicles are gonna move?
How do we think the other agentsin the scene, the humans,
bicycles, how do we thinkthey're all gonna move as time
progresses?
The next major stage isplanning.
So, given that view of theworld, given our predictions of
(13:40):
how things are gonna move, whatshould our vehicle do?
And the goal of planning is touh generate a trajectory.
Now, internally, you'll generatea whole bunch of candidate
trajectories and you'll scorethem, but ultimately you want to
decide on one trajectory tofollow.
So that is a description of yourpath through space.
Then you go to uh motioncontrol, which is turning these
(14:03):
trajectories into commands tothe to the braking, to the
steering, to actually get thevehicle to execute along that
path.
That's the high-level pipeline.
I I have very much simplifiedthis, and you know, those in the
field will know there's a lotmore interplay between these
systems than I've described.
For example, you can't actuallypredict the world without
(14:25):
knowing what you're gonna dobecause other agents are gonna
respond to the path that youplan.
So we're seeing a lot moreblending of these systems, but I
think it's a good high-level wayto understand how we go from
these sensors, which arebasically receiving photons from
the world, and to turn it intoactuation of braking and
steering.
SPEAKER_01 (14:43):
And how much Nikki of this is machine learned, is
is is models.
We've just been talking aboutthat in the last couple of
episodes, but how much is sortof handwritten, sort of symbolic
reasoning and how much of thisis models that you learn from
data?
SPEAKER_02 (14:56):
Yeah, these days there is a a lot of machine
learning and uh you knowartificial intelligence in all
these stages of the system andblending together these stages
of the system.
Now that has evolvedsignificantly, you know, over
the uh 10 plus years that thatWaymo has been working at this
problem.
It's been really accelerated byadvances in AI, right, and and
(15:17):
moving uh more of these systemsto to data-driven, machine
learned-based approaches.
SPEAKER_00 (15:22):
And Nick, would you train the software or the
systems behind it differentlyfor like London versus Phoenix?
Because Phoenix was this testcase city, right?
Because it was super optimizedfor autonomous vehicles, or do
you just train it withinformation about everything
that you can?
SPEAKER_02 (15:38):
Yeah, that's a fantastic question.
Um, these models do tend togeneralize very well over
different cities.
These artificial intelligenceapproaches, these
machine-learned approaches, theytend to generalize well as long
as you have generalized datathat you're feeding into them.
And this is actually one of thebig advantages of them over the
more legacy approach, which is alot of handcrafted software.
(16:00):
That tends to be a disaster whenyou want to go from one city to
the next city.
Because, you know, if you couldimagine, if anyone's somewhat
familiar with programming, uh,one of the basic constructs is
like an if statement.
Okay.
If this condition, then do this,if this conditioned, then do
this.
That and that works fine forvery simple scenarios.
Maybe you could craft somethingthat could work in in some
(16:22):
driving in environment, somesimple driving environment that
way, but then it doesn'tgeneralize that the moment you
see something new, you have torevisit all of those if
statements.
There's just too manypermutations of complexity.
SPEAKER_00 (16:33):
This is an awesome callback, Hugh, to the AIF ceres
that we've just run throughbecause we talk about the
limitations of the human layingout all of the different
scenarios and parameters thatyou want to make a decision on.
And actually at some point, youjust like hand it over to the
technology and say, you're goingto figure this out much better
than we could.
SPEAKER_02 (16:49):
Yeah.
And you hand it over to thesemodels, right?
Where you're fundamentallychanging your programming
paradigm from I'm going tospecify, you know, what to do in
every exact circumstance to amachine learned model where it's
all about training it withhigh-quality data sets and then
evaluating the outputs of thosemodels in a way that is well
(17:13):
correlated with good driving.
Okay.
So this brings its ownchallenges.
You know, you need significantdata.
You need to be able to evaluatehow well it is driving, but it
allows you to generalize, right?
And it's what's going to enableus to bring Waymo to cities like
London and other places acrossthe world without that much
incremental work.
SPEAKER_01 (17:33):
So you've got all this data from all the sensors
on the on the device, and youknow, you're recording that.
So you've got an enormous amountof training data, but what role
do humans sort of play in thatloop?
So are you are you labelingoutcomes as good driving and bad
driving?
I mean, how how does how doesthe human play a role in the
case?
SPEAKER_02 (17:47):
Yeah, that's a that's a really good question.
Um, we do need humans to do someof the evaluation, right?
And to label examples of gooddriving, bad driving, or label
examples of of a scenerepresentation, you know, if
perception outputs a particulardetection of a vehicle, you
know, you want a human to doublecheck was that correct, or or
maybe do some of the labeling inthe first place to bootstrap the
(18:09):
training.
Now, what you quickly find isthat doesn't scale, where you
have you know humans doing everysingle eval and labeling task.
You know, I mentioned we do 100million miles of real world
driving.
Now, in simulation, we've donetens of billions of miles.
So what you end up doing isbuilding a lot of software to do
more of this labeling taskautomatically and more of the
(18:29):
evaluation task automatically.
Now, humans still play a role toquality check those algorithms
and models that are that aredoing the evaluation.
And so what's really interestinghere is uh there's both the
models that are running on thevehicle to actually generate a
trajectory and and drive.
And then there's what we'redoing off the vehicle in
simulation and also in you knowpost-processing of real miles to
(18:54):
evaluate what is good driving,to sort of critique our driving.
And there's models involvedthere as well, you're right.
In fact, even more powerfulmodels to critique the drive
because you have the luxury oftime, you're no longer latency
sensitive.
The whole stack is really movingquickly to these machine learned
approaches.
Humans are needed to label andto evaluate, but you try and
automate that as much as you canas well.
SPEAKER_00 (19:15):
And so is this a fair metaphor?
Like when I think about whatmakes a good driver or someone
that I'd feel safe driving with,one of the things is like they
have a lot of miles under theirbelt, they've got lots of
experience on the road.
And with all of these realdriving hours and billions of
simulated driving hours, we'rebasically giving the system more
experience driving than like anysingle human could obviously
ever have in their entire life.
(19:36):
And that's what makes it safe orsafer.
SPEAKER_02 (19:39):
Yes.
I I believe the average humandoes what, like one million
miles of driving in their acrosstheir lifetime.
So we're over over a hundredhumans worth of accumulated
driving.
And then it's also what Imentioned earlier, which is the
computers don't get distracted.
They never get tired, they neverfall asleep, they're never
looking at their phones, they'repaying attention all the time,
and then they have the benefitof this huge accumulation of
(20:02):
driving data to train.
SPEAKER_00 (20:04):
Forgive me as a lay person and a lay person who
doesn't really drive because Ilive in London.
But I feel like my biggesttakeaway so far is that I had
always thought that inautonomous vehicles, like it's
the hardware that's reallyimpressive, or it's like a big
hardware revolution.
And I'm sure there's some veryimpressive hardware in there,
but it's sounding really likeit's a huge software-driven
product and it's all that smartin the system, or what's really
(20:25):
the revolution here.
SPEAKER_02 (20:26):
Yeah, it's it's definitely both.
We have some quite brillianthardware engineers.
We have optical, mechanical,electrical, manufacturing.
And what the hardware does is itmakes the software problem
easier.
For example, with LiDAR, you canget a precise distance to a
particular object, you know,down to the millimeter
resolution.
(20:46):
With camera, you could get anestimate of distance, but you
have to infer it.
You'd have to see successiveframes of motion or have you
know multiple camera images thatyou're doing, you know,
stereofocusing.
You can use techniques to infer,but it's going to come with
higher errors, and that makesthe software problem more
challenging.
So hardware has really helpedmake the software problem more
tractable because this is thisis really a tough engineering
(21:09):
challenge.
I mean, uh I've been at it foreight years.
Waymo, I think, has been at itfor 15 years or so now.
And you know, the industry hasbeen talking about doing
autonomous driving going onsince the 1980s, actually.
There were some initial efforts.
So it's it's an incredibly toughchallenge.
So anytime that you can makethings a bit easier in hardware,
you know, that's that's often agood choice.
(21:30):
And over time, you know, weexpect the software to be able
to reduce the constraints onhardware and then allow us to,
you know, down cost the vehiclesand simplify things while
retaining the safety andperformance.
SPEAKER_00 (21:42):
That makes sense.
So it sounds like you must havea huge wealth of types of
engineers that work for Waymowith like all of them, their own
individual craft.
And if everything is optimizedto be as high-tech or as sort of
cutting edge as it can be,there's a holistic benefit of
the code.
SPEAKER_02 (21:58):
We do, I absolutely love it.
I've always been a softwareengineer that works close to the
hardware.
You know, I worked on Androidfor a long time.
And I love building somethingthat you can actually touch.
You know, these these arerobots, these are robotic
systems that are extraordinarilycomplicated.
And, you know, I love visitingthe manufacturing lines and
seeing the work that we'redoing.
Uh, I love talking to, yeah, wehave materials engineers, we
(22:20):
have tire experts, you know, wehave battery experts.
And the things you can learnabout how you can we optimize
charging, for instance, bydischarging and recharging at
the optimal rates.
I love being a software engineerworking amongst all of these uh
the these hardware disciplines.
SPEAKER_00 (22:36):
Yeah, that sounds pretty awesome.
I'm really outing myself here assomeone who doesn't know much
about cars, but if you've gotall these sensors and radars and
like cameras and the car getsdirty or like a bird poops on
it, sorry, that's a really grossexample, but I'm just thinking
about it.
Like, what happens if the LiDARdoesn't get affected by that or
(22:57):
like the camera must do, right?
So, how does that all getmanaged in practical terms?
SPEAKER_02 (23:00):
Yeah, no, these are these are great questions.
And this we're driving at suchscale, this is happening not
just every day, but every hour.
You can imagine a like a plasticbag is blown up off the road and
then and covers the aperture ofa camera.
So the yeah, these things arevery real, they do happen.
We have redundancy, is the biganswer.
Any one sensor can completelyfail.
(23:20):
And usually the way it fails isnot a hardware failure, it's
usually environmental, as yousuggested.
It's it's yeah, bird poo on theaperture.
But we have enough redundancythat we can keep driving even
with you know one or moresensors dropping out.
And now what we do is is we'lladjust to those conditions.
So we may drive at a lowerspeed.
If it's severe enough, we mayreturn to base as quickly as
(23:43):
possible.
Uh, in some cases, you know,we'll have to pull over to the
side of the road.
But uh, you know, we haveredundancy to be fault
operational for all these sortsof things.
This is part of the challenge,right?
Of making of deploying thesesystems at scale.
You need to have systems thatkeep driving for across a huge
range of faults.
SPEAKER_01 (24:01):
Can I take us back to the choice of Phoenix and the
choice of San Francisco?
Seems like a good time to sortof talk about the environment.
So, you know, what made Phoenixsort of a great test case for
for Waymo and then and then whydid you choose San Francisco
next?
SPEAKER_02 (24:15):
So the Phoenix choice predated me joining
Waymo, but uh, you know, I Iimagine it was sort of seen as a
sweet spot of okay, there's somechallenges there, there's
there's a reasonable businessthere, but it's also achievable.
This is back sort of 2015, 2016,I imagine these decisions were
made because it's a suburbanlayout.
(24:36):
It doesn't have quite thechallenges of a New York or a or
a San Francisco in terms ofdensity of pedestrians and
agents.
So that was a great place to geta system deployed and also build
all of the surrounding piecesthat you need.
This isn't just a self-drivingvehicle that this is an
application, much like the Uberapp, to hail vehicles.
This is an operations componentto it to charge the vehicles,
(24:57):
clean them.
So there's all these othercomponents to building a
ride-haling ecosystem thatPhoenix gave us an opportunity
to really develop.
But then San Francisco was asignificant jump and it was very
deliberate to go there next toreally pick a city that's
significantly more challenging.
In fact, it's really a sort ofstep function above Phoenix.
(25:18):
In suburbs, you can kind ofdrive on rails and sort of nudge
for okay, if someone'sencroaching on the shoulder, you
sort of nudge around them, butfor the most part, you're
driving within the marked lanes.
As soon as you go to a densecity, London, San Francisco, you
are very frequently needing togenerate quite creative routes
(25:39):
amongst the traffic.
You know, you'll encounterdouble-parked vehicles and you
have to decide, am I going totry and overtake it?
Uh, what route will I pick toovertake it without causing
gridlock?
Or you'll be at an intersectionwhich is just completely in
gridlock, and you have a shortgreen, and you have to decide,
am I going to try and pushacross the intersection to the
other side, or am I going tojust stick here?
(25:59):
Uh, you really have to behave alot more human-like.
So this was a significantchallenge and really required
bringing a lot more AI andmachine learning to the planning
side of things.
SPEAKER_00 (26:10):
Are there cities, Nick, that you just can't see
Waymo ever wanting to try andtackle?
Like I'm thinking Manila orMumbai, where it's just pretty
chaotic and unexpected thingshappen, and maybe the weather is
a bit trickier.
What's the view on that?
SPEAKER_02 (26:27):
At this stage, no, I don't think any city is off
limits because of the advancesin using AI in planning, that
we're able to navigate thesevery complicated scenes.
Now, there'd be more trainingrequired for a city like Manila.
There's just a more diversity ofobjects on the road, a different
style of driving that'srequired.
There'd probably be somedifferent evaluation that's
(26:48):
needed to score what is gooddriving and what is bad driving.
But that's an achievable goalnow.
I was going to just mentionweather as well.
So it's notable that right nowthe cities we drive don't handle
a lot of snow and ice.
But we have announced that weare launching in cities that do
have winter snow and ice.
That is a problem we've knownhas been on the horizon.
(27:10):
And now we have some of theright techniques.
Software is very much involved,but when it comes to weather,
hardware is also a big part ofthe solution.
SPEAKER_00 (27:18):
That's pretty awesome.
And I guess without stating theobvious, there's obviously the
what is technically possible.
And it sounds like there's asoul for many different things.
But then there's like what's thecommercial attraction of X City?
Where do you think you'llresonate best with consumers,
that sort of thing?
And that must play into how youchoose which city Waymo, you
know, launches in next.
SPEAKER_02 (27:38):
Yeah, I mean, fundamentally, we're we're a
business that has spent a fairbit on RD and we need to make
that money back, make it aviable business because we can't
scale unless it's a viablebusiness.
So as a ride hailing business,you look at the markets where
other ride hailing companies aremaking a lot of their money, and
those are typically greatmarkets for Waymo as well.
SPEAKER_00 (27:55):
Is there anything that's much more difficult to
solve for than we as lay peoplemight think?
Because I've taken away thatsafety is actually much easier
than the average person off thestreet would think.
But is there anything that'ssort of the inverse?
Like it should you'd think itshould be super easy and it
turns out to be hard.
SPEAKER_02 (28:12):
Safety is easy if that's all that you cared about,
because you would just park thevehicle and it wouldn't move or
it would creep forward at a veryslow speed and you know stop if
a butterfly was uh seen at 100meters.
But now what's what's hard is tosolve for safety in combination
with you know behavinghuman-like, you know, being a
predictable road user andgetting you to your destination
(28:35):
on a reasonable time.
Uh what you end up doing as youzoom into the different models
involved is you're often makingtrades on what we call the
precision recall curve.
So I mean let's take a sort ofsimple example.
Uh let's say we're looking atthe perception system and it's
trying to make sure it'sdetecting cones, traffic cones
(28:57):
placed in front of it.
So precision would mean that ifit thinks it sees a cone, then
it's really a cone.
Okay, that it's not a tree, thatit's not some other object.
It's precise that that isdefinitely a cone.
Now, recall is that you didn'tmiss a cone.
There was a cone in front ofyou, and the and the perception
system fails to detect it, whatwe call a false positive.
(29:18):
Uh trying to tune these curves,trying to tune systems to
produce precision recall curveswhere we can pick a good
operating point on there thatallows us to detect what's in
front without ever missinganything and never get confused
and think that there's somethingthere that's not, so we don't
make forward progress isextraordinarily hard.
(29:38):
And the cone example is kind ofsimple, but there's much more
subtle examples when you canthink of vegetation, you think
of debris on the roads, youthink of steam coming out of
subway vents, where it's veryhard to, you know, at scale get
the right trade between makingsure you always see something
that you should stop for andthat you don't unnecessarily.
(30:00):
stop for something that youshould proceed through.
And when you get the tradewrong, it could be dangerous in
both directions.
Okay.
If you don't see something, youcould have a collision.
If you think you see somethingthat wasn't really there, you
hit the brakes too hard and thenyou have a tailgator collision.
So it's it's dangerous eitherway.
SPEAKER_01 (30:13):
And is it is it fair, Nick, to to talk about
sort of recall precision in oneway or does it vary across the
systems?
Because I imagine I mean likeGoogle search obviously is a
high precision system, right?
You run a query and you justwant great answers.
You don't necessarily want allthe answers.
I imagine Waymo could be a highrecall system in that you boy,
you better, you better see allthe objects.
But if you think a cone is a asmall tree, is that okay?
(30:35):
So or or is there differenttypes of curves across the whole
system?
How does it actually work?
SPEAKER_02 (30:40):
So definitely the the optimal places on those
curves and the targets for yourlevel of recall, your level of
precision would vary by whatsubsystem, you know, I gave an
example on perception that wehave similar curves looking at
the output of planner, forexample.
Yeah, we're obviously far moresensitive to missing a human
than we would be to missing atraffic cone, for instance.
(31:01):
So this is not a single kind ofglobal optimization.
This is a huge number of metricsand scenarios that we're having
to reason about.
SPEAKER_00 (31:12):
The example that gets cited or a version of the
example that gets cited when youtalk to people sort of off the
street about autonomous vehiclesis like, oh well, what if it has
to choose between like hittingan old lady and hitting you know
a parent with a child.
I think you're going to justtell me that that's not how it
works.
But I know that that will besort of front of mind for
listeners.
SPEAKER_02 (31:30):
Yeah, this is the classic sort of framing where
there's an implication thatthere's a moral judgment being
made by the programmer and thena sort of well you know who's
making that moral judgment.
But it's not actually how thesystem gets built.
I described earlier you knowusing if statements to build a
self-driving system.
And maybe in that kind ofconstruction you would have such
(31:51):
a determination being made.
SPEAKER_00 (31:53):
Right.
Because you could say if it'ssmall, avoid it to avoid like
children.
But then you're I don't knowyou're biased against taller
people, which is problematic indifferent ways, right?
SPEAKER_02 (32:02):
Right, right.
So how these systems areactually built is as I mentioned
with with with these models thatare trained on large amounts of
data.
So it's really about you knowevaluating the the best outcomes
given different data sets.
And the sort of artificialexample of do I drive into the
old lady or do I drive into thesmall child?
(32:24):
It's a sort of a contrivedexample that doesn't really play
out in a in a data set.
What you see in a data set is arich scene with many other
options available and theevaluation would be prioritizing
well you avoid both of themright but it it's just a sort of
overly simplified scene thatdoesn't it doesn't really
reflect the data that we trainon.
There are many other optionsavailable to you know have have
(32:46):
evasive maneuvers.
SPEAKER_00 (32:48):
Okay so Nick before we wrap up the technical section
of this episode I like to ask isthere anything that's cool or
particularly insightful orinteresting for the listeners
that I've not asked because Iwouldn't want to limit us to the
question list that I came upwith before the episode if
there's something interestingbeyond that.
SPEAKER_02 (33:04):
Yeah I mean to me this is just one of the most
interesting engineeringchallenges of our time because
of the well the the complexitybut the breadth of engineering
involved and I've touched onsome of the the hardware side of
things but you know also on thesoftware side we have these
real-time safety criticalsystems on the vehicle as well
as you know rider experience andyou know user interface systems
(33:27):
in the vehicle then we have thethe mobile app and we have a lot
happening in the cloud uhthere's both the sort of ride
hailing system that you canthink about matching demand and
supply and having a efficientmarketplace there, you know,
much like other ride hailingcompanies do, but then also the
simulation systems and the logreplay and the ability to
(33:47):
visualize and play back what'shappened in the field.
There's such a rich ecosystem oftooling and infrastructure off
the vehicle as well.
I don't know if I've ever workedon a project with such a span of
different software systems.
It kind of brings every singlesoftware discipline together as
well as many different hardwaredisciplines.
For those thinking about youknow next steps in in in career
(34:09):
I think autonomous vehicles area fascinating one.
And although we're at thisinflection point where the
technology is starting to workreally well and starting to
scale there's still so much leftahead.
You know this reminds me ofworking on Android in about 2009
when it was just starting tohockey stick you know we were
just starting to see the thesales really follow exponential
(34:31):
curves.
And I remember at the timepeople on the team thinking
like, oh are we done like we'vekind of built most of this now
like and and it's working butare we done?
But no there's always more todo.
There's always more to make itmore efficient to work in more
environments to improve the userexperience to make the system
more efficient and bring costsdown.
You know I I think we'veactually only just begun on this
(34:53):
journey and there's so much moreinteresting work ahead.
SPEAKER_00 (34:56):
It's like cutting edge but then if we think about
what Android probably lookedlike in 2009, oh my goodness,
isn't it great that we didn'tstop there.
SPEAKER_02 (35:04):
I think the Android team in 2009 was maybe around
200 people and that team now isin the in the tens of thousands.
So the bulk of the work is isactually yet to come.
You know we're moving from thephase where it's a bit more RD,
a bit more like will this work?
Which is which to me is superexciting.
Yeah I I like working on thingswhere there's a chance it won't
(35:26):
work.
But now we're transitioning fromthat into you know it does work
and now how well can we make itand how far can we scale it?
SPEAKER_01 (35:32):
This is a very foreign concept I think to
Australians.
I don't think there's too manyAustralians who probably driven
in a Waymo or any autonomousvehicle but I imagine that's not
the case in Phoenix and not thecase in San Francisco.
So those populations must havereally adapted to Waymo.
So maybe maybe you can tell us alittle bit more about that.
SPEAKER_02 (35:48):
Yeah I think we're at this really interesting point
where in the cities where wehave high density which is San
Francisco and Phoenix this is amainstream product you know if
you talk to people in thosecities they know Waymo they use
Waymo many people using it veryregularly it's really ingrained
in their lives yet for the restof the world probably seems like
(36:08):
a foreign concept something thatthey read about and doesn't it
doesn't seem that real to themthat is going to change rapidly
over the next one two and threeyears as this rolls out to more
and more major cities you knowas as we've announced London.
So I I think this is a such aninteresting moment in time where
it's it's mainstream to a smallnumber of people but will very
(36:29):
quickly become well known Ithink in five years' time
everyone will be looking backand won't remember a time before
we had autonomous vehicles likethe smartphone is is today just
ubiquitous.
SPEAKER_01 (36:39):
Do you think mixed cities change when that happens
so you know I can imagine in thelimit there are no cars operated
by humans or all cars aredriverless.
I guess that'll probably neverhappen completely but in the
limit does this changecongestion, city design, those
kinds of things?
SPEAKER_02 (36:54):
Yeah it it it absolutely will um smartphones
were able to roll out veryquickly because they're they're
easier to manufacture that thisis going to take a little longer
for all the impacts to to playout because we're talking about
city design we're talking aboutmanufacturing of much larger
objects we're talking aboutsafety critical system that you
(37:14):
know has a lot of engagementwith regulators as well but this
is the direction it won't justbe ride hailing it'll be
personal car ownership it'll betrucking it'll be all forms of
transportation over time you canimagine some quite high
percentage of cities right nowis dedicated to to vehicles and
especially to parking I believeit's in in the 30 to 40% range
(37:35):
if you look at a city by realestate is dedicated to parking
wow and you know with autonomousvehicles better utilization of
the vehicles so there's lessparking and when you do need to
park them you can easily movethem out outside of the city.
So what this will mean for howcities are laid out and and real
estate is is quite dramatic andthis will be significant.
(37:58):
Also to people's lives I thinkthis will make the world feel
smaller, you know, much like thejet plane did or the automobile
did originally it'll becomeeasier to get from A to B
because you can use that timemuch more productively and you
can know you're getting theremuch more safely.
And I'm sure over time we'll seea sort of abundance of options.
You could imagine autonomousvehicles that have have beds
(38:21):
that you can sleep in.
I I don't know what timelinethat is on, but that's that's
clearly you know where we'regoing that you can work, you
could play, you could sleep inin in these cities and just make
the world feel feel smaller.
So it's very exciting to beworking on something that's
gonna you know will is going tohave this impact over time.
SPEAKER_01 (38:37):
And you do you think they'll be sort of more
swarm-like too I mean like youknow obviously we've talked
about a car and its hardware andits software and decisions it
makes but I guess in a worldwhere there's lots of driverless
cars they can all communicatewith each other and take
advantage of each other'ssensors and there could even be
a standard where you knowdifferent companies share data
and whatever else do you thinkthis becomes sort of more
swarm-like in some interestingway as well do we need traffic
(38:58):
lights?
SPEAKER_02 (38:59):
I mean it's lots of things are going to change right
yeah I ironically this getseasier as more vehicles are
autonomous because some of theharder challenges are dealing
with unpredictable human-drivenvehicles if they're more
autonomous that then then thisbecomes easier and and then you
can take some of thoseopportunities you can imagine on
highways platooning the vehiclessuch that they are far more fuel
(39:21):
efficient because they are justsitting in each other's draft or
yeah not needing traffic lightsbecause they are just
communicating with each other asthey approach the lights and
just you know in a sort ofbeautiful dance sort of
synchronizing theirtrajectories.
Now this is going to take sometime you need to get to some
critical mass where the vastmajority of vehicles are
autonomous for this to reallywork.
(39:42):
And that's why you know as Isaid earlier so much of this
work is yet to come.
We have the basic autonomoussystem functional but so many of
these benefits that we can thatwe can gain are still ahead of
us.
SPEAKER_01 (39:54):
So Nick I know we're getting pretty close to time but
I we haven't spoken much aboutyou and and your role at Waymo
so you're you're a director atWaymo.
So what actually are youpersonally working on?
SPEAKER_02 (40:02):
Yeah so I actually bridge both the hardware and
software organizations I reportinto both the VP of hardware and
the VP of software where I spenda lot of my time recently is on
our reliability systems, ourbackup systems.
You know when something goeswrong making sure that both the
hardware and software are faultoperational.
(40:23):
This is actually reallyimportant for freeways.
We're doing a lot of testing onthem.
We hope to bring them to thepublic very soon what's hard
about the freeway is you need tomake sure when you're at high
speed that if anything goeswrong, the vehicle behaves
safely so this is where thebackup systems become really
critical and we have some quitean amazing capabilities now in
(40:44):
terms of significant parts ofthe vehicle can fail.
It could be for a softwarereason it could be for a
hardware reason it could be anenvironmental reason and the
vehicle will continue drivingand you know exit the dangerous
environment like the freeway andget onto a safer environment.
So yeah I love kind of workingat the at the blending of
hardware and software to makethose sorts of responses
(41:06):
possible.
SPEAKER_00 (41:07):
It sounds like Nick you love to find the most
difficult problem ever to solveand spend your time thinking
through the best way of solvingit.
SPEAKER_02 (41:15):
Yeah I I spend a fair bit of time thinking about
what is the worst thing thatcould happen.
What is the worst scenario thevehicle could find itself in and
how can we make sure the vehicledoes the right thing in that
really bad scenario.
SPEAKER_00 (41:26):
Amazing so that probably brings us to time I'm
very much looking forward tochecking in in one to two years
on whether or not Waymo's takenover the world I'll certainly be
opting in early in London assoon as I can to give it a try.
SPEAKER_02 (41:40):
Fantastic thanks so much for hosting me Hugh and
Hannah it's been been a pleasureand yeah I look forward to to
bringing Waymo to to London andmore places.
SPEAKER_01 (41:48):
Yeah awesome great to see you Nick thanks for your
time so that was awesome Nick isclearly super smart like somehow
one foot in hardware and onefoot in software which from what
I understand is no mean feat andan awesome build on episodes
gone before yeah it was and I'dI'd say to all of our listeners
you know go back if you haven'tand listen to our episodes on AI
apps and coding because they'rejust such a fantastic foundation
(42:10):
for the conversation we had withDick today and really build your
understanding of this wholebroad ecosystem but what a great
guy fantastic.
SPEAKER_00 (42:17):
Yeah and he definitely I know we talked
about London a lot but Nick atleast as far as I could tell was
teasing like a true Waymotakeover.
So if you're not in SanFrancisco or London Waymo could
be coming to City New You thatis pure conjecture not
officially signed off by anyoneat Waymo but that's just my own
inkling and what a great episodeto end on which was sort of the
perfect summation of so many ofthe things that we talked about
and also a tease for a lot ofcool stuff that we could cover
(42:39):
in season two.
SPEAKER_01 (42:40):
Well if you enjoyed the Tech Overflow podcast you
can learn more about us attechoverflowpodcast.com makes
sense as a URL I guess and uhwe're also available on LinkedIn
where uh I post more frequentlythan Hannah does on Instagram
and X.
SPEAKER_00 (42:53):
I actually posted to our Instagram story twice this
week but he's not wrong andleave us a review recommend us
to your friends we really wantto come back for a season two
and that will all be down to youguys.
So do us a solid because Ireally want to go to Melbourne.
See ya.
See you Hannah bye
Hello world and welcome to the Tech Overflow
Podcast.
I'm Hannah Clayton Langton.
SPEAKER_01 (00:09):
And I'm Hugh Williams, and we're the podcast
that explains technical conceptsto really smart people.
How are you, Hannah?
SPEAKER_00 (00:14):
I am great.
I'm on holiday.
I am a little bit disappointedto be back recording remotely,
but such is life.
What about you?
Is the jet lag still veryprevalent?
SPEAKER_01 (00:24):
Oh, the jet lag's super prevalent at uh Melbourne
to London, London to Melbournein a week thing uh really's
really turns you upside down.
We like the most jet set podcastever, because I think you're in
upstate New York.
SPEAKER_00 (00:33):
Is that right?
I am indeed in New Yorkrecording from my friend's guest
room.
So yeah, I don't go very manyplaces without my mic these
days.
SPEAKER_01 (00:41):
And then when this trip's over, you're done for the
year.
Are you gonna stay in London?
SPEAKER_00 (00:44):
Oh no, I think we're going to Paris at some point,
but that's just on the train.
So it doesn't really count, doesit?
SPEAKER_01 (00:52):
One of the cool things about Europe, I mean you
can just say stuff like that.
I'm popping over to Paris.
Uh that's not a concept forAustralians.
We don't pop anywhere.
SPEAKER_00 (00:58):
It's a good segue into our transport-based topic
for today, which is trulyaccidental, but I'm gonna sort
of stretch it to make it work.
So who've we got on today, Hugh?
SPEAKER_01 (01:07):
We have Nick Pelle, who is a director of engineering
at Waymo.
And for those of you who don'tknow Waymo, Waymo is uh a
company that's very well knownto people who live in Phoenix,
in Arizona, in the US, and tofolks who live in San Francisco
in California.
And it's a company that uhdevelops driverless cars, so
autonomous vehicles.
So vehicles you can actuallyjust hop into, they have no
(01:29):
driver, no human supervision,and they'll take you like a robo
taxi from A to B.
And Nick's somebody I knew uhfrom when I worked at Google
Hannah Awesome.
SPEAKER_00 (01:37):
Yeah, I love this phrase robo taxi, which isn't
something that I had heardbefore we started recording.
Um I learned a ton, and I thinkyou guys are going to enjoy it
too.
So we will get right into it.
SPEAKER_01 (01:46):
Yeah, awesome.
Let's get Nick on the show.
Nick Pelly, welcome to the TechOverflow podcast.
It's great to see you, myfriend.
Great to see you here.
Thanks for hosting me today.
SPEAKER_00 (01:56):
Yeah, we're really excited to have you here.
So today we are here to talkabout Waymo and autonomous
vehicles, which, by purecoincidence, since you agreeing
to do the podcast, Waymo haveannounced their London launch,
and we have a good deal of UKand London-based listeners.
So this is really exciting forus.
SPEAKER_02 (02:13):
Yeah, I'm I'm super excited to see Waymo expanding
to more cities and operating infive cities in the US.
And it's you know quite wellknown to those who are uh in
those cities and using theservice.
But to go international, to goto London, I've got family in
London too.
I'm really excited to bring thistechnology to more of the world.
SPEAKER_00 (02:34):
I think it's one of those things that it's really
hard to imagine until youexperience it.
So maybe we kick off with justlike bringing it to life.
Like, what's the customerexperience like of getting in a
Waymo?
Because I have to say, I've beenin an autonomous vehicle, which
we might talk about later, butI've not been in a Waymo.
SPEAKER_02 (02:50):
Yeah, there's there's definitely a variety of
vehicles with autonomousfeatures out there.
But let me describe what Waymois.
It's a robotaxi experience whereyou know you pull out your
phone, fail a vehicle, you know,much like you would do with
Uber.
In fact, in some places wepartner with Uber, and you know,
the vehicle comes to pick youup.
But in this case, the vehicle isempty.
(03:11):
It's gonna pull over on the sideof the road with no one inside.
You get a private vehicle, aprivate experience, and it's
going to fully autonomouslydrive you to your destination.
So this is a ride hailingservice.
It's what in the industry wedescribe as level four autonomy.
Uh, you know, if you'd likelater, we can get into some of
those different levels.
The level four meaning, youknow, there is no human in the
(03:33):
vehicle that is ready to takecontrol of the steering wheel.
There's no one in the vehiclewho needs a driver's license.
SPEAKER_01 (03:39):
I remember when I was at Google, they had those
little uh Waymo bubble cars.
It looks like things haveevolved a lot since then.
So, what's the actual hardwarethese days?
What sort of what sort of a carare we hopping into?
SPEAKER_02 (03:49):
The uh vehicles that we have today, the base vehicle
is a is a Jaguar, Jaguar iPace,but it's been significantly
retrofitted by us to add all ofthe sensors that we need for
autonomous driving.
Cameras, radars, LIDAR, inertialmotion units, IMUs, so that it
can see the world very clearly.
(04:12):
We will be introducing othervehicles, uh, you know, several
that are in testing at thisstage that we hope to bring you
know to the public soon.
SPEAKER_00 (04:19):
And if I'm to put my product management hat on for a
minute, can we start with likethe why?
So, what is the promise of thesedriverless cars?
Why do we need them other thanbeing able to play music and not
having to speak to the Uberdriver when we're going from A
to B?
SPEAKER_02 (04:33):
That's the most noticeable uh for the user is
you get a private experience,and that's a really big win.
But what we think a lot about isthe 40,000 road deaths just in
the US, 40,000 road deaths peryear that are completely
avoidable.
And the safety benefits that anautonomous vehicle bring are
quite dramatic.
(04:54):
We've now driven over a hundredmillion fully autonomous miles.
And you know, we've looked backat that data, and we have
between five and 10x lessaccidents than human drivers who
uh are driving in the samegeographies.
The safety benefit is is reallyquite stark, as well as the
convenience that you mentioned.
SPEAKER_00 (05:15):
That's really interesting because whenever I
mention autonomous vehicles toanyone I know, whether it's
friends or family, admittedly,no one's really in the industry,
but their first sort ofhesitation always comes down to
safety.
So maybe later in the episode wecan get more into like why it's
actually more safe and provingitself to be, because it's very
(05:36):
counterintuitive, at least ifyou don't understand the tech
behind it and why it might besuperior to human drivers.
SPEAKER_02 (05:41):
Yeah, I think that's many people's reactions.
But what we observe is evenafter the first ride, people can
appreciate the safety very, veryquickly because within a few
minutes of driving, you will seehow confident the vehicle is and
it's seeing things, it doesn'tget distracted, right?
It's looking 360 degrees all thetime.
And folks report feeling thatconfidence even within the first
(06:05):
few minutes of riding.
SPEAKER_00 (06:06):
Interesting.
So I think I mentioned justbefore that I've not been in a
Waymo, but if we can cast ourminds back to like 2015, so I
went in an autonomous vehicle inAbu Dhabi in this place called
Mazdar City.
And I remember it having quitesignificant expectations of what
a driverless vehicle would belike because my parents were
living out there and they said,Oh, you know, when you come and
(06:28):
visit, we'll take you in thesesuper cool autonomous vehicles.
And when I went and actuallyrode in one, it was kind of more
like a little shuttle ratherthan like a vehicle that
actually was a car without adriver.
So is that like the earlyiteration of the technology?
Was this sort of shuttleservice?
And now has it evolved pastthat?
SPEAKER_02 (06:46):
Yeah, maybe let me lay out the capabilities a
little bit.
Um, I think this shuttleservice, did it have a driver
inside?
SPEAKER_00 (06:53):
There was no driver, no.
SPEAKER_02 (06:54):
Okay.
So the SAE have levels ofautonomy for vehicles from
what's the SAE?
SPEAKER_00 (07:00):
Sorry, Nick.
SPEAKER_02 (07:01):
The Society of Automotive Engineers.
SPEAKER_00 (07:03):
Okay.
SPEAKER_02 (07:04):
So what you experienced there was a level
four uh autonomous vehicle, andthat's the same as what Waymo
produces level four.
So level one is a is a manuallydriven vehicle.
Level two are vehicles that havesome autonomous features that
require constant humansupervision.
And this is actually very commonin any vehicle that you buy
today.
This is things like cruisecontrol, lane keep assist.
(07:26):
You know, they can significantlyreduce the load on the driver
because many tasks are beingassisted by the vehicle, but the
driver has to fully superviseand be prepared to take over at
any time.
Um then you have level three.
SPEAKER_01 (07:40):
Nick, sorry, with with level two, so you know,
you've got things like adaptivecruise control and the you know,
the lane guidance and things.
Does does Tesla's fullself-driving mode kind of fit
into level two?
So really you've got a humanthere who's supervising whatever
the car's capable of?
SPEAKER_02 (07:54):
Yep, exactly.
I'd say Tesla's FSD is a is avery capable level two.
It actually provides prettysignificant assistance, but but
it's still in the level twocategory.
And that's where there's anotherdimension that I was going to
get to in a moment.
You know, you've got the levelsof autonomy, which is really
about how much supervision isneeded.
And then you've got, you know,what is the operational design
(08:15):
domain?
The the ODD is the terminologywe use for those autonomy
features.
Just how much are they capableof?
So the shuttle that you werereferencing, Hannah, in Abu
Dhabi was level four, but with avery restricted operational
design domain.
It's probably just doingpoint-to-point service.
I imagine this is an environmentthat is not dense with people,
(08:37):
with other traffic, and it'sprobably limited to a modest top
speed.
SPEAKER_00 (08:42):
Exactly.
So if I remember correctly,Mazdar City was almost like a
campus, and you actually parkedyour car like outside and went
in.
The autonomous vehicle didn'teven look or feel like a car
when you rode in it.
It was kind of like this littlepod.
Yep.
And there were no other cars onthe road.
There might have been a fewpeople walking around.
I actually can't rememberbecause it was a while ago.
(09:02):
But it it wasn't replicating thetypical driving experience,
which is interesting becauseit's actually the same level as
Waymo, as you've just sort ofbroken out.
Yeah.
SPEAKER_02 (09:10):
That's right.
It's level four, but but with amore constrained ODD.
And this kind of gets to beginsto get to the heart of you know
what is so odd about building anautonomous system.
You know, you can get to levelfour on a constrained ODD by,
you know, just being verycautious, you know, limiting the
(09:30):
top speed.
And I'd imagine that vehiclewould react pretty quickly to
any object in its presence,right?
If it sees anything, it could bea pedestrian, it would just come
to a stop and not make forwardprogress until it was quite sure
the path is clear.
And that works okay inenvironments that aren't that
dense, that are relativelystructured.
That's not going to work if youput it into the middle of a busy
(09:52):
city like San Francisco orLondon, because it's not going
to make forward progress.
There's going to be too muchactivity, too much going on.
It can't reason about thisenvironment in a sophisticated
enough way to both be safe andto make forward progress in the
environment.
SPEAKER_00 (10:07):
Okay.
So in order to make forwardprogress safely, the
self-driving system, I guess,has to replicate what a driver
would do, or it sounds likemaybe even perform those actions
better, which involves likeperceiving what's around it and
making decisions as a result ofthat.
So how does that all happen?
You mentioned LiDAR.
Um, I'm sure there's a bunch ofother stuff.
(10:27):
Could you talk at a high level,talk the listeners through how
that all hangs together?
SPEAKER_02 (10:31):
Yeah, let me give a high-level overview of a typical
autonomous vehicle um drivingstack, right?
And then we can come back andkind of dig into why this is so
hard to be both safe and for thevehicle to you know make
progress in complicatedsituations.
So at the high level, you've gotall the sensors mentioned
earlier, cameras, radars,lidars, um, audio.
(10:51):
We also have microphones.
SPEAKER_00 (10:53):
Can I ask you a quick question?
Sorry to jump in.
The difference between a radarand a lidar, could you just talk
the listeners through that?
SPEAKER_02 (10:58):
Radars are working on sending out radio frequency
pulses and you know receivingthe returns.
And they are fantastic forlooking for objects that are
reflective to the RF frequency,which is typically metallic
objects, and they give you avery good speed indication.
You get this Doppler return.
That's why police use radarspeed guns.
(11:20):
LIDARs are actually sending outlight pulses instead.
LIDARs are the rotating objectsthat you see on top of the Waymo
vehicles.
You know, they really stand outvisually.
They're very sophisticatedpieces of equipment that are
sending out laser pulses andthen receiving the returns.
And they give very accuratemeasurement of distance to an
object.
SPEAKER_00 (11:40):
So is there like an interplay there between the
lidar knowing the distance andthe radar knowing the speed?
And then like somewhere in thesystem, a calculation of how
likely a collision is.
SPEAKER_02 (11:50):
Yes, and a sort of simplified version of how this
works, and is maybe how thingsworked 10 years ago, is you
would use your lidars to figureout that there is some object in
front of you.
You would then go to thecameras, which are very good,
they're very high resolution,and they can help identify what
the object is.
And then the radar provides theadditional, okay, how fast is
(12:13):
that object moving?
Now that's a very simplifiedversion of how things work.
What happens in the more modernsystems is that all these data
points are fused together.
So you could imagine you haveall of this rich information
coming from lidars, radars,cameras, and very early in the
pipeline.
So before you've even determinedthat there is an object, you're
(12:34):
actually fusing all theinformation together.
The strengths of radar, thestrength of LIDAR, the strength
of camera.
Modern AI allows you to reallyblend those very early in the
pipeline.
And that's at the perceptionphase, which is the next stage
of the pipeline after thesensors.
Perception is consuming, youknow, this is this very high
bandwidth data from the from thecameras, radars, lidars, and
(12:56):
forming a view, like a semanticview of the world.
Okay.
What does the roadway look like?
What are the vehicles?
What are their speeds?
What are their directions oftravel?
Pedestrians, other agents, youknow, cyclists, uh, people on
scooters, people on bicycles,static objects.
So it's it's building a semanticrepresentation of the world.
(13:18):
From perception, we then go toprediction because you need to
then play that representationforwards.
Okay, how do we think thevehicles are gonna move?
How do we think the other agentsin the scene, the humans,
bicycles, how do we thinkthey're all gonna move as time
progresses?
The next major stage isplanning.
So, given that view of theworld, given our predictions of
(13:40):
how things are gonna move, whatshould our vehicle do?
And the goal of planning is touh generate a trajectory.
Now, internally, you'll generatea whole bunch of candidate
trajectories and you'll scorethem, but ultimately you want to
decide on one trajectory tofollow.
So that is a description of yourpath through space.
Then you go to uh motioncontrol, which is turning these
(14:03):
trajectories into commands tothe to the braking, to the
steering, to actually get thevehicle to execute along that
path.
That's the high-level pipeline.
I I have very much simplifiedthis, and you know, those in the
field will know there's a lotmore interplay between these
systems than I've described.
For example, you can't actuallypredict the world without
(14:25):
knowing what you're gonna dobecause other agents are gonna
respond to the path that youplan.
So we're seeing a lot moreblending of these systems, but I
think it's a good high-level wayto understand how we go from
these sensors, which arebasically receiving photons from
the world, and to turn it intoactuation of braking and
steering.
SPEAKER_01 (14:43):
And how much Nikki of this is machine learned, is
is is models.
We've just been talking aboutthat in the last couple of
episodes, but how much is sortof handwritten, sort of symbolic
reasoning and how much of thisis models that you learn from
data?
SPEAKER_02 (14:56):
Yeah, these days there is a a lot of machine
learning and uh you knowartificial intelligence in all
these stages of the system andblending together these stages
of the system.
Now that has evolvedsignificantly, you know, over
the uh 10 plus years that thatWaymo has been working at this
problem.
It's been really accelerated byadvances in AI, right, and and
(15:17):
moving uh more of these systemsto to data-driven, machine
learned-based approaches.
SPEAKER_00 (15:22):
And Nick, would you train the software or the
systems behind it differentlyfor like London versus Phoenix?
Because Phoenix was this testcase city, right?
Because it was super optimizedfor autonomous vehicles, or do
you just train it withinformation about everything
that you can?
SPEAKER_02 (15:38):
Yeah, that's a fantastic question.
Um, these models do tend togeneralize very well over
different cities.
These artificial intelligenceapproaches, these
machine-learned approaches, theytend to generalize well as long
as you have generalized datathat you're feeding into them.
And this is actually one of thebig advantages of them over the
more legacy approach, which is alot of handcrafted software.
(16:00):
That tends to be a disaster whenyou want to go from one city to
the next city.
Because, you know, if you couldimagine, if anyone's somewhat
familiar with programming, uh,one of the basic constructs is
like an if statement.
Okay.
If this condition, then do this,if this conditioned, then do
this.
That and that works fine forvery simple scenarios.
Maybe you could craft somethingthat could work in in some
(16:22):
driving in environment, somesimple driving environment that
way, but then it doesn'tgeneralize that the moment you
see something new, you have torevisit all of those if
statements.
There's just too manypermutations of complexity.
SPEAKER_00 (16:33):
This is an awesome callback, Hugh, to the AIF ceres
that we've just run throughbecause we talk about the
limitations of the human layingout all of the different
scenarios and parameters thatyou want to make a decision on.
And actually at some point, youjust like hand it over to the
technology and say, you're goingto figure this out much better
than we could.
SPEAKER_02 (16:49):
Yeah.
And you hand it over to thesemodels, right?
Where you're fundamentallychanging your programming
paradigm from I'm going tospecify, you know, what to do in
every exact circumstance to amachine learned model where it's
all about training it withhigh-quality data sets and then
evaluating the outputs of thosemodels in a way that is well
(17:13):
correlated with good driving.
Okay.
So this brings its ownchallenges.
You know, you need significantdata.
You need to be able to evaluatehow well it is driving, but it
allows you to generalize, right?
And it's what's going to enableus to bring Waymo to cities like
London and other places acrossthe world without that much
incremental work.
SPEAKER_01 (17:33):
So you've got all this data from all the sensors
on the on the device, and youknow, you're recording that.
So you've got an enormous amountof training data, but what role
do humans sort of play in thatloop?
So are you are you labelingoutcomes as good driving and bad
driving?
I mean, how how does how doesthe human play a role in the
case?
SPEAKER_02 (17:47):
Yeah, that's a that's a really good question.
Um, we do need humans to do someof the evaluation, right?
And to label examples of gooddriving, bad driving, or label
examples of of a scenerepresentation, you know, if
perception outputs a particulardetection of a vehicle, you
know, you want a human to doublecheck was that correct, or or
maybe do some of the labeling inthe first place to bootstrap the
(18:09):
training.
Now, what you quickly find isthat doesn't scale, where you
have you know humans doing everysingle eval and labeling task.
You know, I mentioned we do 100million miles of real world
driving.
Now, in simulation, we've donetens of billions of miles.
So what you end up doing isbuilding a lot of software to do
more of this labeling taskautomatically and more of the
(18:29):
evaluation task automatically.
Now, humans still play a role toquality check those algorithms
and models that are that aredoing the evaluation.
And so what's really interestinghere is uh there's both the
models that are running on thevehicle to actually generate a
trajectory and and drive.
And then there's what we'redoing off the vehicle in
simulation and also in you knowpost-processing of real miles to
(18:54):
evaluate what is good driving,to sort of critique our driving.
And there's models involvedthere as well, you're right.
In fact, even more powerfulmodels to critique the drive
because you have the luxury oftime, you're no longer latency
sensitive.
The whole stack is really movingquickly to these machine learned
approaches.
Humans are needed to label andto evaluate, but you try and
automate that as much as you canas well.
SPEAKER_00 (19:15):
And so is this a fair metaphor?
Like when I think about whatmakes a good driver or someone
that I'd feel safe driving with,one of the things is like they
have a lot of miles under theirbelt, they've got lots of
experience on the road.
And with all of these realdriving hours and billions of
simulated driving hours, we'rebasically giving the system more
experience driving than like anysingle human could obviously
ever have in their entire life.
(19:36):
And that's what makes it safe orsafer.
SPEAKER_02 (19:39):
Yes.
I I believe the average humandoes what, like one million
miles of driving in their acrosstheir lifetime.
So we're over over a hundredhumans worth of accumulated
driving.
And then it's also what Imentioned earlier, which is the
computers don't get distracted.
They never get tired, they neverfall asleep, they're never
looking at their phones, they'repaying attention all the time,
and then they have the benefitof this huge accumulation of
(20:02):
driving data to train.
SPEAKER_00 (20:04):
Forgive me as a lay person and a lay person who
doesn't really drive because Ilive in London.
But I feel like my biggesttakeaway so far is that I had
always thought that inautonomous vehicles, like it's
the hardware that's reallyimpressive, or it's like a big
hardware revolution.
And I'm sure there's some veryimpressive hardware in there,
but it's sounding really likeit's a huge software-driven
product and it's all that smartin the system, or what's really
(20:25):
the revolution here.
SPEAKER_02 (20:26):
Yeah, it's it's definitely both.
We have some quite brillianthardware engineers.
We have optical, mechanical,electrical, manufacturing.
And what the hardware does is itmakes the software problem
easier.
For example, with LiDAR, you canget a precise distance to a
particular object, you know,down to the millimeter
resolution.
(20:46):
With camera, you could get anestimate of distance, but you
have to infer it.
You'd have to see successiveframes of motion or have you
know multiple camera images thatyou're doing, you know,
stereofocusing.
You can use techniques to infer,but it's going to come with
higher errors, and that makesthe software problem more
challenging.
So hardware has really helpedmake the software problem more
tractable because this is thisis really a tough engineering
(21:09):
challenge.
I mean, uh I've been at it foreight years.
Waymo, I think, has been at itfor 15 years or so now.
And you know, the industry hasbeen talking about doing
autonomous driving going onsince the 1980s, actually.
There were some initial efforts.
So it's it's an incredibly toughchallenge.
So anytime that you can makethings a bit easier in hardware,
you know, that's that's often agood choice.
(21:30):
And over time, you know, weexpect the software to be able
to reduce the constraints onhardware and then allow us to,
you know, down cost the vehiclesand simplify things while
retaining the safety andperformance.
SPEAKER_00 (21:42):
That makes sense.
So it sounds like you must havea huge wealth of types of
engineers that work for Waymowith like all of them, their own
individual craft.
And if everything is optimizedto be as high-tech or as sort of
cutting edge as it can be,there's a holistic benefit of
the code.
SPEAKER_02 (21:58):
We do, I absolutely love it.
I've always been a softwareengineer that works close to the
hardware.
You know, I worked on Androidfor a long time.
And I love building somethingthat you can actually touch.
You know, these these arerobots, these are robotic
systems that are extraordinarilycomplicated.
And, you know, I love visitingthe manufacturing lines and
seeing the work that we'redoing.
Uh, I love talking to, yeah, wehave materials engineers, we
(22:20):
have tire experts, you know, wehave battery experts.
And the things you can learnabout how you can we optimize
charging, for instance, bydischarging and recharging at
the optimal rates.
I love being a software engineerworking amongst all of these uh
the these hardware disciplines.
SPEAKER_00 (22:36):
Yeah, that sounds pretty awesome.
I'm really outing myself here assomeone who doesn't know much
about cars, but if you've gotall these sensors and radars and
like cameras and the car getsdirty or like a bird poops on
it, sorry, that's a really grossexample, but I'm just thinking
about it.
Like, what happens if the LiDARdoesn't get affected by that or
(22:57):
like the camera must do, right?
So, how does that all getmanaged in practical terms?
SPEAKER_02 (23:00):
Yeah, no, these are these are great questions.
And this we're driving at suchscale, this is happening not
just every day, but every hour.
You can imagine a like a plasticbag is blown up off the road and
then and covers the aperture ofa camera.
So the yeah, these things arevery real, they do happen.
We have redundancy, is the biganswer.
Any one sensor can completelyfail.
(23:20):
And usually the way it fails isnot a hardware failure, it's
usually environmental, as yousuggested.
It's it's yeah, bird poo on theaperture.
But we have enough redundancythat we can keep driving even
with you know one or moresensors dropping out.
And now what we do is is we'lladjust to those conditions.
So we may drive at a lowerspeed.
If it's severe enough, we mayreturn to base as quickly as
(23:43):
possible.
Uh, in some cases, you know,we'll have to pull over to the
side of the road.
But uh, you know, we haveredundancy to be fault
operational for all these sortsof things.
This is part of the challenge,right?
Of making of deploying thesesystems at scale.
You need to have systems thatkeep driving for across a huge
range of faults.
SPEAKER_01 (24:01):
Can I take us back to the choice of Phoenix and the
choice of San Francisco?
Seems like a good time to sortof talk about the environment.
So, you know, what made Phoenixsort of a great test case for
for Waymo and then and then whydid you choose San Francisco
next?
SPEAKER_02 (24:15):
So the Phoenix choice predated me joining
Waymo, but uh, you know, I Iimagine it was sort of seen as a
sweet spot of okay, there's somechallenges there, there's
there's a reasonable businessthere, but it's also achievable.
This is back sort of 2015, 2016,I imagine these decisions were
made because it's a suburbanlayout.
(24:36):
It doesn't have quite thechallenges of a New York or a or
a San Francisco in terms ofdensity of pedestrians and
agents.
So that was a great place to geta system deployed and also build
all of the surrounding piecesthat you need.
This isn't just a self-drivingvehicle that this is an
application, much like the Uberapp, to hail vehicles.
This is an operations componentto it to charge the vehicles,
(24:57):
clean them.
So there's all these othercomponents to building a
ride-haling ecosystem thatPhoenix gave us an opportunity
to really develop.
But then San Francisco was asignificant jump and it was very
deliberate to go there next toreally pick a city that's
significantly more challenging.
In fact, it's really a sort ofstep function above Phoenix.
(25:18):
In suburbs, you can kind ofdrive on rails and sort of nudge
for okay, if someone'sencroaching on the shoulder, you
sort of nudge around them, butfor the most part, you're
driving within the marked lanes.
As soon as you go to a densecity, London, San Francisco, you
are very frequently needing togenerate quite creative routes
(25:39):
amongst the traffic.
You know, you'll encounterdouble-parked vehicles and you
have to decide, am I going totry and overtake it?
Uh, what route will I pick toovertake it without causing
gridlock?
Or you'll be at an intersectionwhich is just completely in
gridlock, and you have a shortgreen, and you have to decide,
am I going to try and pushacross the intersection to the
other side, or am I going tojust stick here?
(25:59):
Uh, you really have to behave alot more human-like.
So this was a significantchallenge and really required
bringing a lot more AI andmachine learning to the planning
side of things.
SPEAKER_00 (26:10):
Are there cities, Nick, that you just can't see
Waymo ever wanting to try andtackle?
Like I'm thinking Manila orMumbai, where it's just pretty
chaotic and unexpected thingshappen, and maybe the weather is
a bit trickier.
What's the view on that?
SPEAKER_02 (26:27):
At this stage, no, I don't think any city is off
limits because of the advancesin using AI in planning, that
we're able to navigate thesevery complicated scenes.
Now, there'd be more trainingrequired for a city like Manila.
There's just a more diversity ofobjects on the road, a different
style of driving that'srequired.
There'd probably be somedifferent evaluation that's
(26:48):
needed to score what is gooddriving and what is bad driving.
But that's an achievable goalnow.
I was going to just mentionweather as well.
So it's notable that right nowthe cities we drive don't handle
a lot of snow and ice.
But we have announced that weare launching in cities that do
have winter snow and ice.
That is a problem we've knownhas been on the horizon.
(27:10):
And now we have some of theright techniques.
Software is very much involved,but when it comes to weather,
hardware is also a big part ofthe solution.
SPEAKER_00 (27:18):
That's pretty awesome.
And I guess without stating theobvious, there's obviously the
what is technically possible.
And it sounds like there's asoul for many different things.
But then there's like what's thecommercial attraction of X City?
Where do you think you'llresonate best with consumers,
that sort of thing?
And that must play into how youchoose which city Waymo, you
know, launches in next.
SPEAKER_02 (27:38):
Yeah, I mean, fundamentally, we're we're a
business that has spent a fairbit on RD and we need to make
that money back, make it aviable business because we can't
scale unless it's a viablebusiness.
So as a ride hailing business,you look at the markets where
other ride hailing companies aremaking a lot of their money, and
those are typically greatmarkets for Waymo as well.
SPEAKER_00 (27:55):
Is there anything that's much more difficult to
solve for than we as lay peoplemight think?
Because I've taken away thatsafety is actually much easier
than the average person off thestreet would think.
But is there anything that'ssort of the inverse?
Like it should you'd think itshould be super easy and it
turns out to be hard.
SPEAKER_02 (28:12):
Safety is easy if that's all that you cared about,
because you would just park thevehicle and it wouldn't move or
it would creep forward at a veryslow speed and you know stop if
a butterfly was uh seen at 100meters.
But now what's what's hard is tosolve for safety in combination
with you know behavinghuman-like, you know, being a
predictable road user andgetting you to your destination
(28:35):
on a reasonable time.
Uh what you end up doing as youzoom into the different models
involved is you're often makingtrades on what we call the
precision recall curve.
So I mean let's take a sort ofsimple example.
Uh let's say we're looking atthe perception system and it's
trying to make sure it'sdetecting cones, traffic cones
(28:57):
placed in front of it.
So precision would mean that ifit thinks it sees a cone, then
it's really a cone.
Okay, that it's not a tree, thatit's not some other object.
It's precise that that isdefinitely a cone.
Now, recall is that you didn'tmiss a cone.
There was a cone in front ofyou, and the and the perception
system fails to detect it, whatwe call a false positive.
(29:18):
Uh trying to tune these curves,trying to tune systems to
produce precision recall curveswhere we can pick a good
operating point on there thatallows us to detect what's in
front without ever missinganything and never get confused
and think that there's somethingthere that's not, so we don't
make forward progress isextraordinarily hard.
(29:38):
And the cone example is kind ofsimple, but there's much more
subtle examples when you canthink of vegetation, you think
of debris on the roads, youthink of steam coming out of
subway vents, where it's veryhard to, you know, at scale get
the right trade between makingsure you always see something
that you should stop for andthat you don't unnecessarily.
(30:00):
stop for something that youshould proceed through.
And when you get the tradewrong, it could be dangerous in
both directions.
Okay.
If you don't see something, youcould have a collision.
If you think you see somethingthat wasn't really there, you
hit the brakes too hard and thenyou have a tailgator collision.
So it's it's dangerous eitherway.
SPEAKER_01 (30:13):
And is it is it fair, Nick, to to talk about
sort of recall precision in oneway or does it vary across the
systems?
Because I imagine I mean likeGoogle search obviously is a
high precision system, right?
You run a query and you justwant great answers.
You don't necessarily want allthe answers.
I imagine Waymo could be a highrecall system in that you boy,
you better, you better see allthe objects.
But if you think a cone is a asmall tree, is that okay?
(30:35):
So or or is there differenttypes of curves across the whole
system?
How does it actually work?
SPEAKER_02 (30:40):
So definitely the the optimal places on those
curves and the targets for yourlevel of recall, your level of
precision would vary by whatsubsystem, you know, I gave an
example on perception that wehave similar curves looking at
the output of planner, forexample.
Yeah, we're obviously far moresensitive to missing a human
than we would be to missing atraffic cone, for instance.
(31:01):
So this is not a single kind ofglobal optimization.
This is a huge number of metricsand scenarios that we're having
to reason about.
SPEAKER_00 (31:12):
The example that gets cited or a version of the
example that gets cited when youtalk to people sort of off the
street about autonomous vehiclesis like, oh well, what if it has
to choose between like hittingan old lady and hitting you know
a parent with a child.
I think you're going to justtell me that that's not how it
works.
But I know that that will besort of front of mind for
listeners.
SPEAKER_02 (31:30):
Yeah, this is the classic sort of framing where
there's an implication thatthere's a moral judgment being
made by the programmer and thena sort of well you know who's
making that moral judgment.
But it's not actually how thesystem gets built.
I described earlier you knowusing if statements to build a
self-driving system.
And maybe in that kind ofconstruction you would have such
(31:51):
a determination being made.
SPEAKER_00 (31:53):
Right.
Because you could say if it'ssmall, avoid it to avoid like
children.
But then you're I don't knowyou're biased against taller
people, which is problematic indifferent ways, right?
SPEAKER_02 (32:02):
Right, right.
So how these systems areactually built is as I mentioned
with with with these models thatare trained on large amounts of
data.
So it's really about you knowevaluating the the best outcomes
given different data sets.
And the sort of artificialexample of do I drive into the
old lady or do I drive into thesmall child?
(32:24):
It's a sort of a contrivedexample that doesn't really play
out in a in a data set.
What you see in a data set is arich scene with many other
options available and theevaluation would be prioritizing
well you avoid both of themright but it it's just a sort of
overly simplified scene thatdoesn't it doesn't really
reflect the data that we trainon.
There are many other optionsavailable to you know have have
(32:46):
evasive maneuvers.
SPEAKER_00 (32:48):
Okay so Nick before we wrap up the technical section
of this episode I like to ask isthere anything that's cool or
particularly insightful orinteresting for the listeners
that I've not asked because Iwouldn't want to limit us to the
question list that I came upwith before the episode if
there's something interestingbeyond that.
SPEAKER_02 (33:04):
Yeah I mean to me this is just one of the most
interesting engineeringchallenges of our time because
of the well the the complexitybut the breadth of engineering
involved and I've touched onsome of the the hardware side of
things but you know also on thesoftware side we have these
real-time safety criticalsystems on the vehicle as well
as you know rider experience andyou know user interface systems
(33:27):
in the vehicle then we have thethe mobile app and we have a lot
happening in the cloud uhthere's both the sort of ride
hailing system that you canthink about matching demand and
supply and having a efficientmarketplace there, you know,
much like other ride hailingcompanies do, but then also the
simulation systems and the logreplay and the ability to
(33:47):
visualize and play back what'shappened in the field.
There's such a rich ecosystem oftooling and infrastructure off
the vehicle as well.
I don't know if I've ever workedon a project with such a span of
different software systems.
It kind of brings every singlesoftware discipline together as
well as many different hardwaredisciplines.
For those thinking about youknow next steps in in in career
(34:09):
I think autonomous vehicles area fascinating one.
And although we're at thisinflection point where the
technology is starting to workreally well and starting to
scale there's still so much leftahead.
You know this reminds me ofworking on Android in about 2009
when it was just starting tohockey stick you know we were
just starting to see the thesales really follow exponential
(34:31):
curves.
And I remember at the timepeople on the team thinking
like, oh are we done like we'vekind of built most of this now
like and and it's working butare we done?
But no there's always more todo.
There's always more to make itmore efficient to work in more
environments to improve the userexperience to make the system
more efficient and bring costsdown.
You know I I think we'veactually only just begun on this
(34:53):
journey and there's so much moreinteresting work ahead.
SPEAKER_00 (34:56):
It's like cutting edge but then if we think about
what Android probably lookedlike in 2009, oh my goodness,
isn't it great that we didn'tstop there.
SPEAKER_02 (35:04):
I think the Android team in 2009 was maybe around
200 people and that team now isin the in the tens of thousands.
So the bulk of the work is isactually yet to come.
You know we're moving from thephase where it's a bit more RD,
a bit more like will this work?
Which is which to me is superexciting.
Yeah I I like working on thingswhere there's a chance it won't
(35:26):
work.
But now we're transitioning fromthat into you know it does work
and now how well can we make itand how far can we scale it?
SPEAKER_01 (35:32):
This is a very foreign concept I think to
Australians.
I don't think there's too manyAustralians who probably driven
in a Waymo or any autonomousvehicle but I imagine that's not
the case in Phoenix and not thecase in San Francisco.
So those populations must havereally adapted to Waymo.
So maybe maybe you can tell us alittle bit more about that.
SPEAKER_02 (35:48):
Yeah I think we're at this really interesting point
where in the cities where wehave high density which is San
Francisco and Phoenix this is amainstream product you know if
you talk to people in thosecities they know Waymo they use
Waymo many people using it veryregularly it's really ingrained
in their lives yet for the restof the world probably seems like
(36:08):
a foreign concept something thatthey read about and doesn't it
doesn't seem that real to themthat is going to change rapidly
over the next one two and threeyears as this rolls out to more
and more major cities you knowas as we've announced London.
So I I think this is a such aninteresting moment in time where
it's it's mainstream to a smallnumber of people but will very
(36:29):
quickly become well known Ithink in five years' time
everyone will be looking backand won't remember a time before
we had autonomous vehicles likethe smartphone is is today just
ubiquitous.
SPEAKER_01 (36:39):
Do you think mixed cities change when that happens
so you know I can imagine in thelimit there are no cars operated
by humans or all cars aredriverless.
I guess that'll probably neverhappen completely but in the
limit does this changecongestion, city design, those
kinds of things?
SPEAKER_02 (36:54):
Yeah it it it absolutely will um smartphones
were able to roll out veryquickly because they're they're
easier to manufacture that thisis going to take a little longer
for all the impacts to to playout because we're talking about
city design we're talking aboutmanufacturing of much larger
objects we're talking aboutsafety critical system that you
(37:14):
know has a lot of engagementwith regulators as well but this
is the direction it won't justbe ride hailing it'll be
personal car ownership it'll betrucking it'll be all forms of
transportation over time you canimagine some quite high
percentage of cities right nowis dedicated to to vehicles and
especially to parking I believeit's in in the 30 to 40% range
(37:35):
if you look at a city by realestate is dedicated to parking
wow and you know with autonomousvehicles better utilization of
the vehicles so there's lessparking and when you do need to
park them you can easily movethem out outside of the city.
So what this will mean for howcities are laid out and and real
estate is is quite dramatic andthis will be significant.
(37:58):
Also to people's lives I thinkthis will make the world feel
smaller, you know, much like thejet plane did or the automobile
did originally it'll becomeeasier to get from A to B
because you can use that timemuch more productively and you
can know you're getting theremuch more safely.
And I'm sure over time we'll seea sort of abundance of options.
You could imagine autonomousvehicles that have have beds
(38:21):
that you can sleep in.
I I don't know what timelinethat is on, but that's that's
clearly you know where we'regoing that you can work, you
could play, you could sleep inin in these cities and just make
the world feel feel smaller.
So it's very exciting to beworking on something that's
gonna you know will is going tohave this impact over time.
SPEAKER_01 (38:37):
And you do you think they'll be sort of more
swarm-like too I mean like youknow obviously we've talked
about a car and its hardware andits software and decisions it
makes but I guess in a worldwhere there's lots of driverless
cars they can all communicatewith each other and take
advantage of each other'ssensors and there could even be
a standard where you knowdifferent companies share data
and whatever else do you thinkthis becomes sort of more
swarm-like in some interestingway as well do we need traffic
(38:58):
lights?
SPEAKER_02 (38:59):
I mean it's lots of things are going to change right
yeah I ironically this getseasier as more vehicles are
autonomous because some of theharder challenges are dealing
with unpredictable human-drivenvehicles if they're more
autonomous that then then thisbecomes easier and and then you
can take some of thoseopportunities you can imagine on
highways platooning the vehiclessuch that they are far more fuel
(39:21):
efficient because they are justsitting in each other's draft or
yeah not needing traffic lightsbecause they are just
communicating with each other asthey approach the lights and
just you know in a sort ofbeautiful dance sort of
synchronizing theirtrajectories.
Now this is going to take sometime you need to get to some
critical mass where the vastmajority of vehicles are
autonomous for this to reallywork.
(39:42):
And that's why you know as Isaid earlier so much of this
work is yet to come.
We have the basic autonomoussystem functional but so many of
these benefits that we can thatwe can gain are still ahead of
us.
SPEAKER_01 (39:54):
So Nick I know we're getting pretty close to time but
I we haven't spoken much aboutyou and and your role at Waymo
so you're you're a director atWaymo.
So what actually are youpersonally working on?
SPEAKER_02 (40:02):
Yeah so I actually bridge both the hardware and
software organizations I reportinto both the VP of hardware and
the VP of software where I spenda lot of my time recently is on
our reliability systems, ourbackup systems.
You know when something goeswrong making sure that both the
hardware and software are faultoperational.
(40:23):
This is actually reallyimportant for freeways.
We're doing a lot of testing onthem.
We hope to bring them to thepublic very soon what's hard
about the freeway is you need tomake sure when you're at high
speed that if anything goeswrong, the vehicle behaves
safely so this is where thebackup systems become really
critical and we have some quitean amazing capabilities now in
(40:44):
terms of significant parts ofthe vehicle can fail.
It could be for a softwarereason it could be for a
hardware reason it could be anenvironmental reason and the
vehicle will continue drivingand you know exit the dangerous
environment like the freeway andget onto a safer environment.
So yeah I love kind of workingat the at the blending of
hardware and software to makethose sorts of responses
(41:06):
possible.
SPEAKER_00 (41:07):
It sounds like Nick you love to find the most
difficult problem ever to solveand spend your time thinking
through the best way of solvingit.
SPEAKER_02 (41:15):
Yeah I I spend a fair bit of time thinking about
what is the worst thing thatcould happen.
What is the worst scenario thevehicle could find itself in and
how can we make sure the vehicledoes the right thing in that
really bad scenario.
SPEAKER_00 (41:26):
Amazing so that probably brings us to time I'm
very much looking forward tochecking in in one to two years
on whether or not Waymo's takenover the world I'll certainly be
opting in early in London assoon as I can to give it a try.
SPEAKER_02 (41:40):
Fantastic thanks so much for hosting me Hugh and
Hannah it's been been a pleasureand yeah I look forward to to
bringing Waymo to to London andmore places.
SPEAKER_01 (41:48):
Yeah awesome great to see you Nick thanks for your
time so that was awesome Nick isclearly super smart like somehow
one foot in hardware and onefoot in software which from what
I understand is no mean feat andan awesome build on episodes
gone before yeah it was and I'dI'd say to all of our listeners
you know go back if you haven'tand listen to our episodes on AI
apps and coding because they'rejust such a fantastic foundation
(42:10):
for the conversation we had withDick today and really build your
understanding of this wholebroad ecosystem but what a great
guy fantastic.
SPEAKER_00 (42:17):
Yeah and he definitely I know we talked
about London a lot but Nick atleast as far as I could tell was
teasing like a true Waymotakeover.
So if you're not in SanFrancisco or London Waymo could
be coming to City New You thatis pure conjecture not
officially signed off by anyoneat Waymo but that's just my own
inkling and what a great episodeto end on which was sort of the
perfect summation of so many ofthe things that we talked about
and also a tease for a lot ofcool stuff that we could cover
(42:39):
in season two.
SPEAKER_01 (42:40):
Well if you enjoyed the Tech Overflow podcast you
can learn more about us attechoverflowpodcast.com makes
sense as a URL I guess and uhwe're also available on LinkedIn
where uh I post more frequentlythan Hannah does on Instagram
and X.
SPEAKER_00 (42:53):
I actually posted to our Instagram story twice this
week but he's not wrong andleave us a review recommend us
to your friends we really wantto come back for a season two
and that will all be down to youguys.
So do us a solid because Ireally want to go to Melbourne.
See ya.
See you Hannah bye
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Bobby Bones Show
Listen to 'The Bobby Bones Show' by downloading the daily full replay.