5th of November - Spatial Intelligence in AI

In this episode of the AiSultana Podcast, we explore the groundbreaking concept of Spatial Intelligence in AI, featuring insights from Fei-Fei Li and Justin Johnson, co-founders of World Labs. Moving beyond traditional 2D models, they discuss how spatial intelligence—AI's ability to understand and interact with the 3D world—represents the next frontier of technological innovation.

Key topics include the power of 3D representation, advancements in computing, and transformative applications in fields like augmented reality, robotics, and virtual world creation.

What new possibilities will spatial intelligence unlock in our digital and physical worlds?

Brought to you by AiSultana, a consultancy specializing in AI solutions for industry.

Join us daily for concise updates on crucial developments in AI, and why they matter to you.

Available via YouTube, Apple, and Spotify.

Don't forget to like and subscribe, and explore our free wine consumer app at www.aisultana.com.

Tune in to stay informed about the pivotal topics shaping the future of AI in industry.

Episode Transcript

(00:00):
Okay, so get ready because this deep dive,

(00:02):
well, it's going to take us way past just chatbots.
We're stepping into the future of AI,
like way into the future.
Spatial intelligence.
It's not just about AI processing language anymore.
It's about AI understanding the world,
like in 3D, just like we do.
Like teaching a computer to actually see,
not just process an image,

(00:22):
but understand what it's looking at, the depth.
Exactly, yeah.
And to help us wrap our heads around this,
we've got some serious AI rock stars in our corner today.
Fei-Fei Liva and Justin Johnson.
Oh, wow, the World Labs founders.
Yep, the masterminds behind World Labs,
and they've even got Martin Cassado
from A16 STYZ jamming in.
This is big.
What's at you so hype?

(00:43):
It's the way they're framing this whole thing.
They're saying, forget what you thought you knew about AI.
This spatial intelligence, this 3D understanding,
this is the real deal, the real game changer.
So they're saying move over language models,
there's something bigger on the horizon.
Exactly, they're basically saying,
forget the first few chapters of AI,
we're diving straight into the climax.

(01:04):
It's interesting though,
how they bring it back to those early days,
the AI winters.
Oh, right.
Fei-Fei and Justin, they've been around since then,
seeing the rise of deep learning firsthand.
Oh, so they've seen it all, the ups and the downs.
Yeah, and they were a huge part of it.
Remember ImageNet, it's not just some tech term,
that was their baby.
ImageNet, the massive data set of labeled pictures.

(01:26):
That's the one, that data set,
it's like they taught computers how to see.
Yeah, it was a game changer for sure.
But this whole data thing, it's crucial, right?
I mean, these AI models,
they need massive amounts of data to function.
Huge amounts, it's crazy.
They call data the new oil for a reason.
And it's even more true with spatial intelligence.
Think about it, to just identify an object,

(01:48):
you need tons of data, right?
But to then understand its shape, its texture,
how it moves in the world.
You're talking about an entirely different level
of complexity.
And data demands.
And that's where things get really interesting,
because here's where we hit this fork in the road.
You've got language models, powerful as heck,
but they're stuck in this one dimensional world,

(02:11):
the world of words in a line.
I see what you're saying,
but isn't that how we communicate?
Yeah, but the world we live in,
it's anything but a straight line.
It's full of dimensions.
So you're saying language is just one slice of the pie.
It's like, think about it this way,
reading a recipe versus actually baking a cake.
One's information, the other's experience.

(02:32):
Okay, I get you.
So spatial intelligence, it's about bridging that gap.
That's where spatial intelligence comes in.
It's about bridging that gap
between information and experience.
Okay, I like where this is going.
And it makes you think, right?
Like, remember the first time you tried VR, those headsets?
Oh yeah, how could I forget?
It's like you were transported somewhere else.
Spatial intelligence,

(02:52):
it could unlock a whole new level of that,
a whole new world of immersive experiences.
Like, we wouldn't even be able to tell the difference
between digital and physical anymore.
I mean, blurring the lines between the real world
and the virtual world
sounds like something straight out of a movie,
but what would that even look like?
Okay, so we're not just talking about, like,
better video games here.
And that those are definitely coming.

(03:12):
Oh yeah, for sure.
But imagine a world where robots can navigate our homes
as easily as we can, AR.
Forget those clunky overlays.
We're talking seamlessly integrated digital elements
into our world.
And experiences, experiences in 3D, tailored just for you.
It's like stepping into the future.

(03:33):
But what about right now?
Where does World Labs fit into all of this?
Right, so they're on it, and they're not messing around.
Feifei and Justin, they put together this dream team,
bringing in the best of the best,
computer vision, robotics, 3D modeling, you name it,
they've got it covered.
Sounds like they're going all in on this.
They're on a mission to unlock spatial intelligence,
like the full potential of it.

(03:54):
And they say now is the time to do it.
This isn't some far off thing.
They've got these concrete milestones they're aiming for.
So they have a plan, a roadmap to make this happen.
What kind of milestones are we talking about here?
Well, for one, they're talking about creating a system
that can generate complete interactive 3D worlds.
Not just static models,

(04:14):
but worlds you can actually step into and interact with.
Like in real time.
Wow, you're blowing my mind here.
So instead of just creating a 3D model of say, a building,
you'd be able to walk through it, open doors,
turn on lights.
Yeah, like it's the real deal.
Think about the implications for things like architecture,
gaming, it could be huge.
This is bigger than I even imagined.

(04:35):
It's like we're on the verge of something massive.
But hold on a second, this all sounds incredible,
but isn't there a catch?
Building these 3D worlds,
understanding them at that level of detail,
it must require a crazy amount of data, right?
You're hitting on one of their biggest challenges.
It's not just about teaching AI
to recognize an object in a picture.

(04:56):
We're talking about understanding its entire form,
the texture, how it moves in 3D space.
So how do they even begin to tackle that?
It's not like they can create a 3D scan
of the entire planet.
That's where it gets even wilder.
They're looking at creating these massive,
synthetic 3D environments.
Synthetic, like not real.

(05:16):
Exactly.
Think of it like a digital playground.
They can control everything.
How light interacts with objects,
the physics of movement.
It's like they're building a virtual stage
to rehearse for the main event in the real world.
Precisely.
Instead of trying to capture every single detail
of the real world.
Those would be impossible, by the way.
Totally.
They're creating these controlled environments

(05:36):
to train their models.
It's like a blend of computer vision and AI imagination.
It's a fascinating mix.
And it lets them experiment at a scale
that would be impossible in the real world.
I bet.
Build a virtual city one day,
try out different weather conditions the next.
Exactly.
Even populated with AI controlled people in cars.
It's like SimCity,
but for serious scientific advancement.

(05:58):
This is next level stuff.
But how do we know this isn't all just pie in the sky?
Are there any real world examples
of this kind of technology in action today?
Actually, yeah.
Think about those latest, most realistic video games.
Those incredible graphics,
the details, the immersive worlds.
Gamers spend hours in those virtual environments

(06:20):
and they feel real the way you can interact with things.
And those games,
they're built on really sophisticated 3D engines.
The same tech that World Labs things could be used
for spatial intelligence.
So the building blocks are already there.
Exactly.
And it's not limited to just entertainment.
Think architecture, engineering, manufacturing.
They're already using 3D modeling and simulation.

(06:40):
But spatial intelligence,
that takes it up like 10 notches.
It's like we're moving from a world of screens
and keyboards to a world where we can reach into the digital
and manipulate it like it's real.
And that's where that collaborative approach
they're so big on comes in, right?
They're bringing in not just computer scientists,
but robotics experts, even cognitive psychologists.
They know that to build AI that really gets our world,

(07:03):
they need to understand how we experience it as humans.
Makes sense.
But this is a massive undertaking.
How do they even measure success?
It's not like there's a finish line they're gonna cross.
You're right.
They talk about it more in terms of milestones,
not endpoints.
Like one of their first goals is to build a system
that can generate a complete 3D world.

(07:24):
And not just visually,
but one that behaves realistically too.
Okay, so not just a 3D model of a building,
but an environment where you can open doors,
turn on lights, really experience it.
Yes.
And it's not pre-programmed.
Those interactions are driven by the AI's understanding
of how things work in the real world.
That level of fidelity, that interaction,

(07:44):
it's mind blowing.
It's like taking VR to a whole new level.
What if instead of just escaping into these worlds,
we could use this technology to actually shape our own?
Now you're thinking,
it's like we could give architects and engineers
these incredible tools, right?
Let them design and test their ideas in a virtual world
first before building anything in the real one.
Exactly.

(08:04):
No more expensive prototypes
or oops, we forgot something, moments.
And think about education.
Imagine if instead of just reading about ancient Rome,
students could actually walk through the forum,
talk to digital citizens.
That's wild.
History would come alive.
It'd be like stepping right into the past.
It's taking learning to a whole other level,
making it immersive, engaging.

(08:26):
But with all this talk about the amazing things
this technology could do,
do they ever talk about the downsides?
I mean, this is powerful stuff.
Couldn't it be misused?
They definitely address that.
They say any powerful technology has risks,
but they seem really focused on making sure
this is developed responsibly.
Safeguards against bias, things like that.
So they're thinking about the ethical side of it too.

(08:47):
Yeah, they're big on transparency, accountability,
bringing in experts from different fields
to have those tough conversations.
They wanna make sure it benefits everyone.
That's really good to hear
because it's not just about pushing technological boundaries,
it's about making sure those advancements
actually make the world a better place.
That's exactly.
It's been an amazing conversation.
I mean, this whole deep dive, it really makes you think

(09:10):
it's not just about the technology itself,
but how it could change our lives,
how we work, how we learn, everything.
It really is.
It's a story about what humans can do
when they put their minds together, you know?
The creativity, the drive to solve complex problems.
It's inspiring.
And it makes you wonder if we can teach machines
to see the world like we do, what else is possible?

(09:32):
It's a question worth asking.
So to our listeners, we'll leave you with that thought.
What do you think this new era
of spatial intelligence will bring?
Will it be the ultimate challenge for AI?
And what will it mean for our future?
Keep those questions in mind as we keep diving deeper
into these cutting edge technologies.