July 15, 2025 • 46 mins
In this episode of Practical AI, Chris and Daniel explore the fascinating world of agentic AI for drone and robotic swarms, which is Chris's passion and professional focus. They unpack how autonomous vehicles (UxV), drones (UaV), and other autonomous multi-agent systems can collaborate without centralized control while exhibiting complex emergent behavior with agency and self-governance to accomplish a mission or shared goals. Chris and Dan delve into the role of AI real-time inference and edge computing to enable complex agentic multi-model autonomy, especially in challenging environments like disaster zones and remote industrial operations.
Featuring:
Links:
Chris's definition of Swarming:
Swarming occurs when numerous independent fully-autonomous multi-agentic platforms exhibit highly-coordinated locomotive and emergent behaviors with agency and self-governance in any domain (air, ground, sea, undersea, space), functioning as a single independent logical distributed decentralized decisioning entity for purposes of C3 (command, control, communications) with human operators on-the-loop, to implement actions that achieve strategic, tactical, or operational effects in the furtherance of a mission.
© 2025 Chris Benson
Sponsors:
- Outshift by Cisco: AGNTCY is an open source collective building the Internet of Agents. It's a collaboration layer where AI agents can communicate, discover each other, and work across frameworks. For developers, this means standardized agent discovery tools, seamless protocols for inter-agent communication, and modular components to compose and scale multi-agent workflows.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Jerod (00:04):
Welcome to the Practical AI podcast, where we break down
the real world applications ofartificial intelligence and how
it's shaping the way we live,work, and create. Our goal is to
help make AI technologypractical, productive, and
accessible to everyone. Whetheryou're a developer, business
leader, or just curious aboutthe tech behind the buzz, you're
(00:24):
in the right place. Be sure toconnect with us on LinkedIn, X,
or Blue Sky to stay up to datewith episode drops, behind the
scenes content, and AI insights.You can learn more at
practicalai.fm.
Now, onto the show.
Daniel (00:49):
Welcome to another fully connected episode of the
Practical AI Podcast. In theseepisodes where we don't have a
guest, it's just Chris and I, wetake some time to deep dive into
some interesting AI topics orexplore some things in the news
related to AI and hopefully giveyou a little bit of, learning
(01:10):
resources or information to helpyou level up your machine
learning or AI game. I'm DanielWitenack. I am CEO at Prediction
Guard, and I'm joined as alwaysby my cohost, Chris Benson, who
is a principal AI researchengineer at Lockheed Martin. How
are you doing, Chris?
Chris (01:27):
Hey. Doing great today, Daniel. How are you today?
Daniel (01:29):
I'm I'm doing well. Yeah. I just took a tried to
take a run before it getsreally, really hot here. So
hopefully, you know, that thatworked out well. It was still
kind of hot.
Hopefully, I don't end up with asunburn, but
Chris (01:45):
Absolutely. Yeah. It's it's been hot here in The US,
especially in the Southern USwhere we both are right now. And
so yeah. You definitely don'tneed you don't need you
collapsing from heat exhaustionright before we get into a show
here.
Daniel (02:01):
Yeah. Yeah. For sure. I I definitely say there's fewer
people out. There was not in anyway a swarm of people running,
anywhere that I anywhere that Isaw.
So, yeah, I guess that leads alittle bit into what you teed me
up to talk about today, Chris,which I'm excited about, which
is an area of of your expertiseand and interest, which is
(02:25):
swarming, which I guess could berelated you know, people might
be thinking of animal swarming,robot swarming, other swarming.
Maybe just starting there, whatwhat do we mean by swarming?
Chris (02:39):
So before I before I get into that from my side, let me
let me set that up a little bitbecause so because we you
actually mentioned severalthings that knowing me and and
also, you know, just the generaltopic as we've kinda prepped to
do this a little bit. You know,you mentioned animal swarming.
And so it's funny. In the AIworld, we we have been talking
(03:01):
about, you know, robotics andautonomy and, you know, unmanned
aerial and ground vehicles forvarious things. You hear about
Amazon drones and Walmart dronesare out there now and various
other applications.
And so even though my backgroundmay be in the in kind of
military applications becauseI'm in the defense industry,
(03:21):
we're really talking about it ina general say. And and one thing
I I should say upfront to covermyself is I'm only representing
my own personal viewpoint andnot that of my employer or any
other organization.
Daniel (03:35):
Point taken.
Chris (03:36):
Point taken, which is kind of funny because when you
talk about swarming, if you put10 people into a room and ask
them each to define swarming,you'll end up with about 17
different definitions of whatswarming is. It's one of those
types of terms.
Daniel (03:51):
Yeah. You mentioned you mentioned the delivery drones
around where I'm at. I'm I liveright by Purdue University and
actually for quite a few yearsnow we've had these little
little box robots that willdeliver you food. So they, drive
(04:12):
along the sidewalks and, youknow, go through intersections
and deliver food to you, throughthrough an app and and all that
stuff. So that one definitelyhits a little bit close to home.
I think for the most part, Ihave seen them, you know,
successfully navigate theterrain and and intersections
(04:34):
and all of that. Although I willsay I did see one at one point.
I didn't see the the collision,but it was definitely run over
on the on the road. It was inpieces at an at an intersection.
Chris (04:47):
Yeah. Somebody didn't get their dinner that night. That
sounds
Daniel (04:49):
like Exactly.
Chris (04:50):
Somebody called up hungry. Where's my food? You
know? Which, you know, and itand it was the delivery driver
once upon a time. Hopefully,that person would not have
gotten hit.
But, yes, having having the thesome sort of automated vehicle
bringing the stuff. So wanted totalk a little bit about and and
probably before we get into, youknow, talking about what
swarming is, kind ofdistinguishing it because it's
(05:13):
one of those autonomy words and,you know, autonomy is another
one of those words. And we havethings like drone and robot and
UAV, UGV, UXV, x covering kindof all the things. And so
Daniel (05:26):
I don't know what a lot of those things are.
Chris (05:28):
Okay. So so ground ground, sky. You a US phone,
air. Yeah. So like like, UXVwould be unmanned or uncrewed is
a little bit more modern termfor it.
The x signifies kind of what thedomain it's operating in air
ground, whatever, and thenvehicle. And if you've seen u x
(05:50):
s, that would be an unmanned oruncrewed whatever system. And so
those are really common lingothings, and they're not specific
to military. You see those incommercial and industrial
applications and stuff. And forthe most part, we'll probably
for simplicity's sake, mostlytalk about drones to to in this
conversations, we tend to stayaway from jargon, and maybe
(06:13):
robots on, you know, for groundstuff.
And that way, are not trying toget through acronyms while we're
talking and stuff just tosimplify things a bit. But we're
having the we're definitelyliving in this age, as you just
pointed out from personalexperience, where we're we're
starting to see these things,which are, you know, physical
embodiments of some sort of AIor or other algorithmic driven,
(06:36):
you know, movement around thephysical world for various
activities and stuff. And itsounds like and I don't know the
specific technology for the onethat's bringing you the food on
how they're how they'reapproaching that because there's
a bunch of different approaches.But that is a either a semi
autonomous presumably or a fullyautonomous, you know, ground
(06:56):
vehicle that's bringing you thestuff. Whereas going back to our
topic, swarming implies numbers,first of all.
And so, but it doesn't justimply numbers, it implies the
way numbers are working togetherand collaborating. And I think
that's where a lot of people getin trouble with all sorts of
different definitions. And it'ssuper popular to talk about
(07:18):
swarming now.
Daniel (07:19):
Yeah. It's
Chris (07:20):
it's a crazy buzzword these days.
Daniel (07:22):
Yeah. I almost wonder, like, there could be some
confusion. We hear a lot of talkthese days about multi agent
systems. And, of course, we'vehad episodes where we discuss
agents specifically and whatthat term agents mean. But as
soon as you soon as you bring inthat kind of multi agent side,
(07:44):
people might be confused.
Is is that what we're talkingabout? I guess one question
would be like swarming, does itimply like, physical, I guess,
physical AI or physical autonomyin terms of things that are
operating in the physical world,not the digital world? We've
(08:04):
talked a lot about drones androbots in the beginning of this.
So is that is that part of thatdefinition or or not really? It
is.
Okay.
Chris (08:13):
It is. You know, and we've had across a lot of our
episodes recently, especiallyour fully connected episodes
like this where it's you and Idiscussing a topic. You know,
we've talked about we're kindof, you know, we've had over the
history of of the show, we'vesaw seen the evolution of AI,
and there tend to be specifictopics that get really hot. And
and right now agents are reallyhot, but there's also the notion
(08:36):
of agents applying themselvesagain, you know, in
collaboration with various otherAI technologies, whether they be
LLMs or reinforcement learningor, you know, computer vision.
And I think if you take a stepfarther into this world of
autonomy, especially modernautonomy in 2025 agents are a
big part of that, and havingmultiple agents collaborating,
(08:58):
and they're using these othermodels to get these tasks done.
So you have agents operatingwith LLMs for different
purposes. And you know, soyou're able to go and grab the
right model for the right taskand you're putting a bunch of
tasks together to go dosomething which which I might
call a mission, but I don'tnecessarily mean that in a
(09:19):
military esque way. Could be,but it could be something that
you're that is what the purposeof your company's autonomy is to
do. Know? Maybe like
Daniel (09:27):
a a goal or a objective or outcome
Chris (09:31):
That's right.
Daniel (09:31):
Could be synonyms. Maybe I'm I'm not that great at the
English language, but
Chris (09:36):
Those are all those are all loose synonyms that could
they could apply. Yeah. And soand we're seeing this, you know,
you're we're seeing theseindividual drones and robots
that are starting to do tasks inthe commercial and industrial
space. Certainly, it's been thatway in the industrial space and
warehouses and robot. I guessthe first robot I worked on was
(09:58):
2,000 that personally I wasdoing were two of them that were
autonomous in 2018.
So this is not new in thatcapacity. How it's doing it has
changed over time.
Daniel (10:07):
Yeah. Maybe it just taking a moment kind of looking
back a little bit because Iremember what what year was it
that the I forget the year ofthe Beijing Olympics when they
had the It was the first one Ihad seen where they had a kind
(10:29):
of, I guess we could call it aswarm, I don't know, You can
correct me if I'm wrong. Swarmof of drones that were sort of
in the sky in the I guess, Ithink it was the opening
ceremony. I'm I'm trying toremember, but that was the first
time I had seen that. Of course,I've seen it in other places,
you know, after that.
But, but yeah, what's what'skind of the The definition of
(10:52):
that? Yeah. Well, how would youhow would you represent the
history of kind of coming to thepoint where we're at now in
these, so I'm getting from you,there's these autonomous, so
multiple objects in the physicalworld that are autonomous or
multiple vehicles or robots orwhatever in the physical world
(11:15):
that are autonomous trying toaccomplish a goal or mission.
Now in the Beijing case, I don'tknow if those were autonomous or
not. But, yeah, help me kindaparse through maybe a little bit
of that.
Chris (11:27):
So I should say ahead of time that I don't have any
special knowledge of thatparticular configuration. So I'm
making educated guesses on that.Yeah. So I would I would suggest
that's not what I what and I'lldefine a swarm in a moment as a
follow-up. But I would not callthat a swarm.
I would call that a large numberof individual aerial platforms
(11:48):
that are operating in a in apredetermined coordinated
manner.
Daniel (11:52):
Yeah. It's like synchronized
Chris (11:53):
They're synchronized from the ground control system. So
the notion in in flyingautonomous vehicles is that you
have a ground control station,which might be like a laptop, it
might be a bigger thing inside atruck or something, but it has,
you know, comms and it has waysof communicating and and so for
shows like that, you would use,you know, you would use things
(12:16):
like GPS and they would have aeach drone would have a
three-dimensional path that it'sfollowing Mhmm. That's a little
bit different from all theothers. But when you put them
all up there at the same time,they look like a big coordinated
show, but they're not actuallycoordinated. It's the human on
the ground programming in theirpath.
It is
Daniel (12:35):
They are not determining their coordination.
Chris (12:39):
There you go, which is really important. So Okay. Yeah.
Yeah. You went right to theheart of it.
And so there's that, which isreally not intelligent at all.
It's pre programming thesedrones to do something and the
visual impact is that you havethis thing happening among a
bunch of things. But it's, it'ssort of an illusion. It's really
a bunch of individual things ontheir path. And then there's a
(13:01):
whole there's a whole continuumof how those kind of vehicles
can operate.
One kind of popular approachtoday that I that that a lot of
people would call swarming andwhich I don't, and I'll define
it right after I say this, isthe notion of like giving plays
to a group of vehicles that goout and do a task. Sort of like
(13:25):
some people the analogy will bea football play. It's go go to
this location and do this, andthere might be some
communication with the otherplatforms in your area that
you're doing something with, butI also don't consider that a
swarm. Because I think it comesback to the notion of what is
swarming. It's it's swarming isnot an autonomous thing.
(13:48):
It's a type of behaviorexhibited. And we see that in
nature. And so I will I'll givethe definition and then I'm
actually gonna go away fromtechnology for one second and
talk about what we see in mothernature that's kind of consistent
with that. So my definitionsounds a little military like,
but don't take it don't take ittoo much that way. It can be it
(14:11):
could easily be applied tocommercial or industrial.
So but I think it gets the gistof how I of how I see swarming.
Mine is, swarming occurs whennumerous independent, fully
autonomous platforms exhibithighly coordinated locomotive
behaviors in any domain, be itair, ground, sea, undersea,
space, functioning as a single,independent, logical,
(14:35):
distributed, decentralized,decisioning entity for purposes
of command, control, andcommunications with human
operators on the loop toimplement actions that can
achieve strategic, tactical, oroperational effects in the
furtherance of a mission. Littlemilitary sounding, but if you
translate those words intocommercial and industrial, they
(14:57):
still apply. You might use it toyou might choose a synonym in
some of those areas. But that isa is is a pretty high bar right
there.
And, and as we go forward, wecan talk a little bit about kind
of what in that definition, turnsomething from a collection of
autonomous vehicles into a swarmof autonomous autonomous
(15:18):
vehicles.
Sponsor (15:33):
Well, friends, if you want to build the future of
multi agent software, check outAgency. That's AGNTCY. That's an
open source collective buildingthe Internet of agents. It is a
global collaboration layer whereAI agents can discover, connect
and work across frameworks. Thatmeans better tools for you,
(15:54):
standardized agent discovery,seamless inter agent
communication and modularcomponents to scale your multi
agent workflows.
And they're teaming up with CrewAI, Langchain, Cisco and many
more dropping real code specsand services with no strings
attached. Start buildingalongside engineers who care
about high quality multi agentsoftware. Learn more at
(16:17):
agency.org. Again, that'sagntcy.org. That's agency.org.
Daniel (16:28):
Well, Chris, I I really appreciate your definition.
Maybe that's even something wecan pull out and put into the
show notes for people so thatthey can can read that, if they
if they want to. I'm actuallyreminded as we're talking about
this as an and as I'm hearingyour definition of these things,
(16:49):
you know, in the animal kingdom.Actually, I'm remembering a
movie that I watched with mywife, The Murmuring. I think it
was on Netflix and veryinteresting movie, but kind of
at the core of the movie werethese, I don't know if I'm using
the right terminology, butmurmurations of swarms of of
(17:12):
birds.
Star Lake, I'm sure
Chris (17:14):
famous for it.
Daniel (17:15):
Exactly. Yeah. So you can see in the sky, I'm sure
many people have seen thisswarms of birds that almost form
kind of patterns andsynchronizations in the sky and
and move around together. Nowaccording to your definition,
each of those birds, right? NowI guess in this sense, you know,
(17:39):
there's no connection to a,obviously a human command and
control type of scenario, butcertainly they are autonomous,
but they're accomplishingsomething together, which is
maybe a little bit mysterious inthis case, although I I'm sure
it's a topic of study.
(18:00):
But, but, yeah, I I that's whatcame to my mind is is that is
that movie and that kind ofphenomenon.
Chris (18:07):
Yeah. And that murmuration, if you will, is
literally it's called flockingin that case, you know, which is
obviously a bird term, but thatis definitely consistent with
what I would consider to be aswarm. And if you if you break
that down, mean, there's andthere are other animals we can
talk about too, they havedifferent types of swarming
behavior, but that areconsistent with that. If you
(18:29):
talk about starlings inparticular, you know, they're
beautiful when you see thosemassive, and there could be
thousands of starlings movingacross, and you see the waves
within the system, you know,within the flock as they're
doing that. And and so you kindawhat's going on there?
We have thousands of individualbeings that are communicating
(18:50):
and sensing each other. Theyhave a unified purpose. They're
all going someplace together,but there's no there's no one
master general or CEO starlingthat's going, you guys are gonna
go that way. Everyone do what Itell you to do. They are all
subscribing to the mission, ifyou will, but each one has a has
(19:11):
a a position, if you will, or aplace in that mission where they
are communicating and the waythey do their sensing of each
other and their communicationshelps them a not collide and
and, you know, fail by bythrough collisions and such.
It keeps them together, but theyalso have a way of of agreeing,
(19:32):
if you will, to go do something.They're going to to move from
one place on the earth toanother place on the earth. And
and that kinda comes down to thecrux of it, is that swarm
behavior is where you have thata bunch of fully autonomous
beings in this case, they'refunctioning in that murmuration
(19:53):
as a single independent logicaldistributed decentralized
decisioning entity. It's a longmouthful that I use because all
of those have a part in thatdescribing it, but they're all
working together in that way.
Daniel (20:06):
Do you wanna pick apart maybe each of those words? So
just like why why use thosemultiple words kinda
distributed, decentralized? Somepeople might kind of not know
why each of those is isimportant.
Chris (20:20):
Sure. So so single implies that the joint actions
together about how they'recommunicating and sensing give a
rise to one generalized process.
Daniel (20:34):
Kind of an emergent or combined
Chris (20:36):
process. That's a great way of putting it. And that's
kind of that that emergent, youknow, we said mission or goal
kind of thing is coming out ofthat. And that process is
independent of any one bird. Andit's and it's independent of any
controlling authority thatsaying from afar, y'all shall go
do this thing, you know.
And so you have thatindependence, you have that
(20:59):
single emergent characteristic,logical meaning, it's also kind
of, it's kind of think about,it's arising a an emergent
intelligent notion or aprocessing notion that's all
happening by the unity. It isdistributed across the entire
flock or murmuration. It isdecentralized. There's no one
(21:22):
special bird or group of specialbirds that are the birds that
are in charge of everybody elseand you guys have to do what we
say. And finally, it's adecisioning entity that by by
this emerging thing happening,choices are made collectively
for the whole thing.
And yet, no one bird is saying,everyone's going to go do this
(21:42):
thing. So it's all these kindsof characteristics about that
emergent, I may add the wordemergent into the definition
that I have there, because Ithink that's really a great way
of describing that. But it'sthat unified collection to do
this. Now, that's a far cry fromwhat we're currently doing in
the autonomy space at large. I'mnot saying that doesn't exist in
(22:06):
little pockets, academic pocketsand other places, labs, whether
they be government labs oruniversity labs and stuff like
that.
But you don't see widespreaddeployment of drones or robots
that are acting in this manneras we sit here recording today.
Daniel (22:24):
Okay. That that makes sense. Yeah. Yeah. Part of my
what I'm thinking and myquestioning in my mind, which
maybe you're going to get to asyou kind of get get through some
of these things is, obviously,we have a kind of inspiration or
a model there from the theanimal kingdom.
In the case of, let's say,robots in a warehouse or a
(22:49):
manufacturing facility orautonomous vehicles, we have in
probably things that people arefamiliar with, the idea of self
driving things or autonomy. Solike self like the Waymo cars or
that sort of thing. And there'scertainly, obviously, AI at play
(23:09):
there in terms of computervision and maybe reinforcement
learning and decision making. Soassuming that people might know
at least or be exposed to that,in terms of the connection to AI
specifically, what about thiskind of swarming or, you know,
(23:33):
robots or these entities workingtogether to do a task? What is
necessary for AI to do in thatswarming scenario that's not
that's not necessary in just asingle, like, self driving car,
like a Waymo car or somethinglike that?
Chris (23:50):
That's a great question. There's a lot of nuance there.
So so the self driving car ishaving to navigate an
environment in which it's bothsensing and to some degree
communicating potentially,although that is evolving over
time, with with other actors inthat environment around it. And
there is emergent behavior onwhat to do. There's kind of the
(24:14):
classic cases of, you know, kidsplaying on the sidewalk and the
ball bounces into the street,know, and and reacting to
various unexpected events likethat.
But at the end of the day, thatvehicle is still, which may be
fully autonomous, is still onlyhaving to decision on its own
behalf. And so it has an onboardinference system, the onboard
(24:38):
computers, the onboard models,or potentially that in
combination with some remotethat's doing over some sort of
connection, radio connection orcellular connection, but it's
still decisioning for itself.The difference according to my
definition of a swarm is thatthe emergent quality of of the
decisioning is arising fromparticipation in many of the
(25:04):
platforms that are participatingin that unified mission or
activity. But it has to decidefor all of them and it may have
to decide on platform A, we'relooking at something, we're
evaluating it, we're sensing it,we're processing based on
reinforcement learning and ouragents are then saying platform
a, you go do this activity.Platform b, you go do that one.
(25:26):
Platform c, you go do that one.And those are all slightly
different activities as thoseplatforms collaboratively work
together based on the sensingand inferencing that's happened
across the swarm to get anactivity accomplished. Does that
clarify a little bit?
Daniel (25:45):
Yeah. That that helps. And I'm trying maybe this is the
decentralized part of whatyou're talking about, but part
of maybe what is a challenge inmy mind to think about is how
such a thing would happenwithout a central kinda, so
(26:05):
like, so to say like, oh, let'sjust take a very simple maybe
it's simple, you know, in mymind it seems complex as well,
but we have 20 robots in amanufacturing facility trying to
do something. In one model, Icould see where those, each of
those robots detects certainthings on a number of sensors,
(26:30):
maybe cameras or temperaturesensors or force sensors or
something like that. All of thatis communicated back to a
central processing system.
And it's really the singleentity, the kind of master brain
that makes the decision andcommunicates out the next step
to all the robots. But in adecentralized way, you talked
(26:53):
about how there's not this kindof central there may be a
central command where maybethere's oversight or something,
but things somehow happen in adecentralized way. That's what
I'm trying to connect in my inmy brain maybe.
Chris (27:07):
No. That's that's a great nuance that you're pointing out
there. So in the warehouse,that's not really that that's
definitely a centralizedenvironment, and you don't even
have to have onboard compute forthat because your robots can
have a highly reliable system ofcommunication over WiFi, you
know, with a master serverthat's there and you're in the
(27:28):
distances involved or closeenough to where you can have
them interact with each otherwithout having to do that. And
according to the definition ofthe swarm that we're using here,
that would not be a swarm, eventhough you may have many
platforms working
Daniel (27:42):
Multiple devices.
Chris (27:43):
Collaboratively, they're being controlled by by a local
centralized agent. And inswarming, generally, according
to this definition, you'rereally looking at an environment
where that's not possible.You're looking, you know, what
we would traditionally call edgeor or, you know, far edge is
another term that what peoplehave used. Basically, where you
(28:04):
can't count on either local orcloud environments to provide
the compute. And so you have tohave the compute on board,
including all the activitiesthat will arise from that
compute.
Daniel (28:16):
Yeah. I'm almost imagining moving still kind of
in my mind living in theindustrial world if I think more
to like oil and gas and likecertain of the environments that
are part of part of working inthat industry, whether it be
deep under the ocean or in verydisconnected remote places where
(28:42):
you might not even want to exwant to or even could expose
humans to those environments.
Chris (28:49):
That's right.
Daniel (28:50):
But they're also very hard in terms of connectivity
and and that sort of thing. Youmight want to accomplish tasks
in in that sort of environment.Am I getting to the right kind
of scenarios?
Chris (29:01):
Okay. Like, one good one that is commonly used is the
notion of disaster recovery,where you might go into an area
that is geographic, you know, itmay be geographically remote or
may not be, but either way theinfrastructure has likely things
have happened with theinfrastructure to where things
have been torn down, you don'thave cell towers, things like
(29:23):
that, and you don't have thelevel of connectivity necessary
for centralized control. And soif you're thinking about
disaster recovery in a swarmcontext, you where you do not
have that infrastructure inplace to guide those, then you
could put drones or robots onthe ground that have their own
(29:45):
inference capability. And ifyou're approaching it with that
decisioning entity that isdecentralized that I've
described then that could occurwhere they're actually working
collaboratively to save lives.You know if there's buildings in
rubble, know one robot that hasthat ability as maybe pulling
(30:06):
rubble off because it has thatcapability while others that
specialize in getting downbetween the rubble and they're
smaller, might be in otherwords, not all robots may be the
same.
You might have a heterogeneousmix of robots with different
capabilities, but they're ableto dynamically go, here's a rock
and it needs to be moved whileyou go under the rock, you other
(30:27):
platform go under the rocklooking for a survivor there.
And so you can have a collectionof robots that each has an
optimal function. Yeah. Wherethey're working together to do
something and ultimately save alife and you don't have, you
know, maybe you've had hurricanecome through or tornadoes and
you don't have the normaldigital infrastructure that
(30:50):
we're all so used to today.
Daniel (30:52):
Well well, Chris, that that is really helpful to break
down maybe some of the use casesor scenarios that maybe compare
and contrast multiple agents orrobots or systems that are
working in an environment thatmaybe are in a swarm or are not
(31:13):
in a swarm. Maybe part of sothat's helpful on the use case
front. My mind is still kind ofwondering on this side of the AI
side, what unique AI problemsare unique to the swarm
environment that aren'tencountered elsewhere? And how
(31:36):
do those map to kind of maybeopen challenges that are that
are open or kind of maybe knownmodels that are able to handle
certain of these things?
Chris (31:45):
Yeah. So there's I mean, I think you just kind of alluded
to the complexity involved. AndI think the reason we haven't
had swarms before now and thatthey are still something that
that is emerging within what isbeing developed is because not
all of the technologies andmodels that you would need have
(32:05):
been mature enough up until thispoint in in time to be able to
do that. So there's a lot ofpieces. And some of that is on
the AI side and some of that ison is operating in a physical
environment.
And so on the AI side, you know,over the years, we've talked
about these differentarchitectures, you know,
(32:26):
different models with differentpurposes. And swarming ends up
taking quite a few of those andmaking them available to agents
so that you can within acomplicated physical
environment, different agents onplatform are working with each
other to accomplish the onplatform parts, and then be able
(32:47):
to communicate and coordinatethat with other platforms that
also each are multi agent multimodel configurations. And so and
then being able to do that.
Daniel (32:58):
Swarm of swarms.
Chris (32:59):
Yeah, sort of like if you think of all the various
technologies that go into just asingle platform able to do that.
We're only now really gettingyou know, in the large getting
to where these kinds of thingsare possible. And where the
compute is powerful enough foredge devices and that the models
are being reduced. We've talkeda lot over the last year or two
(33:21):
about how models are gettingsmaller and you get a hugging
face and, you know, the vastmajority of models there are
very small models that areuseful in a lot of specific
tasks. And so you have to matchup small models that can run on
edge inference or or even CPUsto do various tasks for specific
agents and coordinate that withsensors and comms.
(33:43):
Yeah. To be able to accomplishstuff. So it's quite
complicated. And that kind ofand I use this kind of in air
quotes, that kind ofminiaturization of the
technology. I don't meanphysical miniaturization, but
kind of getting everything towork and run on a remote edge
device that is that is likelybattery powered is is a bit of a
(34:05):
challenge.
So I think, as we look forwardright now, there's, you know,
we're, we're really at ainflection point in history in
terms of being able to do this.
Daniel (34:14):
Yeah. And what just did a very practical level, I'm sure
there's a great diversity here,but in some of what you're
talking about, there's a uniquecommunication element and goal
setting element of thesesystems. And if I look at like
AI models that are running onany particular platform within
(34:36):
the swarm, what, at a very highlevel, what kinds of models and
tasks need to be accomplishedthat are associated with the
swarming versus just likesensing and that sort of thing?
Are these models that are, youknow, small SVMs or something
(34:58):
that are making a kind of binarydecision or gradient boosting
machines or neural networks orgen AI models? Like what is the
diversity you see and what arethe kinds of tasks you just, by
way of example, a couple of themthat might need to be done?
Chris (35:14):
Yeah. I I it's a mixture of different models, some of
which can be more traditional AImodels like what we most often
talk about here on the show.Some of them are more of the
classical data science modelsbecause, you know, we've talked
about this a number of timeswhere you don't need to go to a
bigger, harder, you know,expensive model to use if a
smaller model will work, butit's very task specific and then
(35:37):
and it's bound within thesoftware systems of each
platform but also the softwaresystems which govern sensing and
comms. And then so you need theability for different nodes, if
you will, of what it you know,different platforms to maybe on
behalf of the swarm to take onsome of the computation because
you can't distribute a modelinference across all of the
(36:00):
things. But what you can do issay, if you have a swarm of a
dozen, I'm just obviously makingthis up off the top of my head,
have three of the platformsrunning inference on a
particular model for aparticular task while three
others do another thing and thenyou can you can have things like
election algorithms which decidewhich one you're gonna take.
(36:22):
And so you can you can haveevaluation of what who's getting
the best sensing information andwho has the best comms reach for
the entire swarm to ensure thatdata gets around. So you can
have some redundancy in thecomputation and all those things
and then by election pick andelection's just one mechanism.
That's one common algorithmicmechanism that that one can
(36:45):
apply. But by election withinthe swarm, choose who is
issuing, you know, the resultsof the various types of
inference that the agents areputting together and deciding
upon a task. So that you canthen go and assign specific
platforms to do different partsof that task, where not every
platform is doing the samething, but they're all
(37:05):
contributing to the larger goal.
Daniel (37:07):
Gotcha. Yeah. That's very helpful. Now I'm sure
there's some people out therethat are maybe excited by this,
disturbed by this topic, maybe.Definitely.
I mean, one sense, just again,thinking about things outside of
the robotic world, we talkedabout the birds, which are
(37:29):
beautiful and you see thepatterns which they form. I also
think about like in the humancontext, there's interesting
maybe swarm behaviors that aresomewhat disturbing. Like if you
think about a mob mentality, youoften hear about people, you
(37:49):
know, maybe a a mob of peopleall accomplish a very disturbing
thing, like a destructive thing.And then afterwards, you know,
individuals that were in thatswarm say, I don't know what,
you know, the individuals mightnot ever have done what was done
by the mob, right? But becauseof this emergent behavior,
(38:14):
things happen that maybe areoutside of the bounds of any
individual's, you know, moralcompass or that sort of thing.
So if we take this then to therobotic or drone or industrial
side, some people might say,well, how do we push one
direction and not the otherdirection? And what is the
(38:34):
oversight related to this?
Chris (38:37):
So it's a great question. I'm glad I'm glad you brought it
up in my enthusiasm for thetechnology. It's very easy to
lose sight of this. And we had avery recent show where we talked
a little bit about, know, theanthropic, I would refer people
back to that. And the fact thatyou had, you know, models that
were coming out with what wedeem to be bad outcomes.
And that can certainly applyhere. And so you have to have
(39:00):
some form of guardrails. And sothere's two, I want to start
with, there's two phrases, oneof which I mentioned before. I
said human operators on theloop. And human operators on the
loop versus human operators inthe loop are two distinct
things.
Right now in the in the militaryspace, based on current
(39:22):
regulations here in The US andstuff, you you most often are
talking about in the loopinteractions between humans and
stuff. But as you go forward,and this is not a military
specific thing, this issomething that you're gonna see
in commercial and industrial aswell, as you start talking about
swarm capabilities with emergentproperties happening, then the
(39:47):
challenge is that there aresometimes you want a human to be
able to step in the loopdirectly into that processing
and say yes, no or make aselection or something like
that. But there are other timeswhen you're talking about a
larger swarm where you need ahuman who where the it may be
there may be too much going onand there may be too many of
(40:08):
these platforms flying around oron the ground for one human to
track everything at every momentand interact but you can have a
human on the loop, can say forthe overall goal or for the
thing the swarm decides approveor disapprove, you know, if
there an emergent behavior comesup, and the swarm makes a
decision based on what it'sseeing in real time, then you
(40:31):
can say, No, that's not going towork.
And that's what a human on theloop does is you can it gives
you that guardrail of saying,that's not the emergent behavior
I want, you know, it's time togo back or maybe do a full
recall and shut down. So I thinkif you really want to learn more
on these kinds of things you'restarting to see I it's, I'm not
(40:52):
sure that I can think of aspecific swarm. I'll have to do
some research and see if there'sany specific swarming classes. A
lot of the kind of drone androbot systems that you can get
out there in open source. Forinstance, there's ROS and ROS
two, which is the roboticoperating system, first and
(41:14):
second version.
ROS two takes a slightlydifferent approach to ROS one,
so there are some people on eachversion. I know ROS two has some
swarming capabilities in thereto build on. It doesn't
instantly give you swarmcapability but it has some tools
in there that you can startbuilding on and that might be a
good place to start to learnabout the topic. But in general,
(41:38):
kind of staying tuned intolearning sources, including this
podcast, where you can learnabout multi agent environments
and as we increasingly aretalking about physical world
deployments of AI technologies,those are all good places to to
consume because this is a verycutting edge area to to be
focused on.
Daniel (41:58):
Yeah. It seems like a lot of a lot of things are
developing, and I personally amvery excited about some of the
things happening in robotics.Obviously, we've maybe some of
us, the more we see chatinterfaces, it's like we just
wanna see some other applicationof of AI technology, which that
(42:21):
is only that's such a smallsegment of what is possible with
AI, even Gen AI, there's so manymore ways that that will, I
think, become embedded in ourphysical spaces. If you just
wanna search for kind of thisidea of physical AI, I know
that's a big topic of kind of AIembedded in our physical spaces,
(42:46):
maybe not through chatinterfaces, but through other
interfaces. And one thing thatI've really enjoyed seeing is
the stuff coming out of, PollenRobotics, p o l l e n.
That's a company that I believeif if I'm not mistaken was
acquired by Hugging Face andthey're releasing a lot of well,
(43:10):
I don't know a lot. They'rereleasing robotic systems that
are more geared towardsexperimentation, open source
development, integration of openmodels, integration of apps and
exchanging of those apps evenwithin Hugging Face spaces but
for actual physical systems.Actually, Prediction Guard, our
(43:32):
company, we got one of the orordered one of these, I think
it's Ricci Mini, robots. Lookingforward to playing around with
that in our office. That shouldbe really fun.
So I think also there's anincreasing number of ways that
people can access even physicalsystems or robots in a much more
(43:57):
accessible way than before wheremaybe everything was coming out
of whatever it was, BostonDynamics or or wherever very,
very expensive millions ofdollars systems here. There's
ways to access these kind ofsystems, whether it's drones or
robots or that sort of thing inin a in a much more accessible
(44:18):
way, which is reallyencouraging.
Chris (44:19):
It's not expensive anymore. So in the maker space,
there is a lot of resources. Alot of it is based on stuff you
may already know, like Arduinoor Raspberry Pi, and you know,
adding a Jetson in their carrierboards where you basically get
some basic stuff. So we're gonnasee more and more of this going
forward. And as people reallykind of get even some of the
(44:41):
more advanced topics figuredout, It's gonna be in all of our
lives in the years ahead.
So we're gonna see a rapidintroduction into daily life of
these things. And so if thistopic interests you, definitely,
I would encourage you just as Ido, is to go out and jump into
the maker space and and for verylittle money, you can have some
(45:03):
pretty interesting experiencesbuilding some of these things.
Daniel (45:06):
Yeah. And I it's occurring to me that after this,
we'll we'll put up, maybe a awebinar, that where we could
talk about these things a littlebit more, maybe even
demonstrating some things withwith robotics. So don't forget
to check out practicalai.fm/webinars for, upcoming times
(45:28):
that that will be live wherediscussions can happen. And,
yeah, just really appreciate youhelping us deep dive into these
subjects, Chris. It's been it'sbeen really good.
Chris (45:38):
It's a fun topic. Happy to do it. Thanks a lot, Daniel.
Daniel (45:41):
Alright. See you soon.
Chris (45:42):
See you later.
Jerod (45:49):
Alright. That's our show for this week. If you haven't
checked out our website, head topracticalai.fm and be sure to
connect with us on LinkedIn, X,or Blue Sky. You'll see us
posting insights related to thelatest AI developments, and we
would love for you to join theconversation. Thanks to our
partner Prediction Guard forproviding operational support
for the show.
(46:09):
Check them out atpredictionguard.com. Also,
thanks to Breakmaster Cylinderfor the Beats, and to you for
listening. That's all for now,but you'll hear from us again
next week.
Welcome to the Practical AI podcast, where we break down
the real world applications ofartificial intelligence and how
it's shaping the way we live,work, and create. Our goal is to
help make AI technologypractical, productive, and
accessible to everyone. Whetheryou're a developer, business
leader, or just curious aboutthe tech behind the buzz, you're
(00:24):
in the right place. Be sure toconnect with us on LinkedIn, X,
or Blue Sky to stay up to datewith episode drops, behind the
scenes content, and AI insights.You can learn more at
practicalai.fm.
Now, onto the show.
Daniel (00:49):
Welcome to another fully connected episode of the
Practical AI Podcast. In theseepisodes where we don't have a
guest, it's just Chris and I, wetake some time to deep dive into
some interesting AI topics orexplore some things in the news
related to AI and hopefully giveyou a little bit of, learning
(01:10):
resources or information to helpyou level up your machine
learning or AI game. I'm DanielWitenack. I am CEO at Prediction
Guard, and I'm joined as alwaysby my cohost, Chris Benson, who
is a principal AI researchengineer at Lockheed Martin. How
are you doing, Chris?
Chris (01:27):
Hey. Doing great today, Daniel. How are you today?
Daniel (01:29):
I'm I'm doing well. Yeah. I just took a tried to
take a run before it getsreally, really hot here. So
hopefully, you know, that thatworked out well. It was still
kind of hot.
Hopefully, I don't end up with asunburn, but
Chris (01:45):
Absolutely. Yeah. It's it's been hot here in The US,
especially in the Southern USwhere we both are right now. And
so yeah. You definitely don'tneed you don't need you
collapsing from heat exhaustionright before we get into a show
here.
Daniel (02:01):
Yeah. Yeah. For sure. I I definitely say there's fewer
people out. There was not in anyway a swarm of people running,
anywhere that I anywhere that Isaw.
So, yeah, I guess that leads alittle bit into what you teed me
up to talk about today, Chris,which I'm excited about, which
is an area of of your expertiseand and interest, which is
(02:25):
swarming, which I guess could berelated you know, people might
be thinking of animal swarming,robot swarming, other swarming.
Maybe just starting there, whatwhat do we mean by swarming?
Chris (02:39):
So before I before I get into that from my side, let me
let me set that up a little bitbecause so because we you
actually mentioned severalthings that knowing me and and
also, you know, just the generaltopic as we've kinda prepped to
do this a little bit. You know,you mentioned animal swarming.
And so it's funny. In the AIworld, we we have been talking
(03:01):
about, you know, robotics andautonomy and, you know, unmanned
aerial and ground vehicles forvarious things. You hear about
Amazon drones and Walmart dronesare out there now and various
other applications.
And so even though my backgroundmay be in the in kind of
military applications becauseI'm in the defense industry,
(03:21):
we're really talking about it ina general say. And and one thing
I I should say upfront to covermyself is I'm only representing
my own personal viewpoint andnot that of my employer or any
other organization.
Daniel (03:35):
Point taken.
Chris (03:36):
Point taken, which is kind of funny because when you
talk about swarming, if you put10 people into a room and ask
them each to define swarming,you'll end up with about 17
different definitions of whatswarming is. It's one of those
types of terms.
Daniel (03:51):
Yeah. You mentioned you mentioned the delivery drones
around where I'm at. I'm I liveright by Purdue University and
actually for quite a few yearsnow we've had these little
little box robots that willdeliver you food. So they, drive
(04:12):
along the sidewalks and, youknow, go through intersections
and deliver food to you, throughthrough an app and and all that
stuff. So that one definitelyhits a little bit close to home.
I think for the most part, Ihave seen them, you know,
successfully navigate theterrain and and intersections
(04:34):
and all of that. Although I willsay I did see one at one point.
I didn't see the the collision,but it was definitely run over
on the on the road. It was inpieces at an at an intersection.
Chris (04:47):
Yeah. Somebody didn't get their dinner that night. That
sounds
Daniel (04:49):
like Exactly.
Chris (04:50):
Somebody called up hungry. Where's my food? You
know? Which, you know, and itand it was the delivery driver
once upon a time. Hopefully,that person would not have
gotten hit.
But, yes, having having the thesome sort of automated vehicle
bringing the stuff. So wanted totalk a little bit about and and
probably before we get into, youknow, talking about what
swarming is, kind ofdistinguishing it because it's
(05:13):
one of those autonomy words and,you know, autonomy is another
one of those words. And we havethings like drone and robot and
UAV, UGV, UXV, x covering kindof all the things. And so
Daniel (05:26):
I don't know what a lot of those things are.
Chris (05:28):
Okay. So so ground ground, sky. You a US phone,
air. Yeah. So like like, UXVwould be unmanned or uncrewed is
a little bit more modern termfor it.
The x signifies kind of what thedomain it's operating in air
ground, whatever, and thenvehicle. And if you've seen u x
(05:50):
s, that would be an unmanned oruncrewed whatever system. And so
those are really common lingothings, and they're not specific
to military. You see those incommercial and industrial
applications and stuff. And forthe most part, we'll probably
for simplicity's sake, mostlytalk about drones to to in this
conversations, we tend to stayaway from jargon, and maybe
(06:13):
robots on, you know, for groundstuff.
And that way, are not trying toget through acronyms while we're
talking and stuff just tosimplify things a bit. But we're
having the we're definitelyliving in this age, as you just
pointed out from personalexperience, where we're we're
starting to see these things,which are, you know, physical
embodiments of some sort of AIor or other algorithmic driven,
(06:36):
you know, movement around thephysical world for various
activities and stuff. And itsounds like and I don't know the
specific technology for the onethat's bringing you the food on
how they're how they'reapproaching that because there's
a bunch of different approaches.But that is a either a semi
autonomous presumably or a fullyautonomous, you know, ground
(06:56):
vehicle that's bringing you thestuff. Whereas going back to our
topic, swarming implies numbers,first of all.
And so, but it doesn't justimply numbers, it implies the
way numbers are working togetherand collaborating. And I think
that's where a lot of people getin trouble with all sorts of
different definitions. And it'ssuper popular to talk about
(07:18):
swarming now.
Daniel (07:19):
Yeah. It's
Chris (07:20):
it's a crazy buzzword these days.
Daniel (07:22):
Yeah. I almost wonder, like, there could be some
confusion. We hear a lot of talkthese days about multi agent
systems. And, of course, we'vehad episodes where we discuss
agents specifically and whatthat term agents mean. But as
soon as you soon as you bring inthat kind of multi agent side,
(07:44):
people might be confused.
Is is that what we're talkingabout? I guess one question
would be like swarming, does itimply like, physical, I guess,
physical AI or physical autonomyin terms of things that are
operating in the physical world,not the digital world? We've
(08:04):
talked a lot about drones androbots in the beginning of this.
So is that is that part of thatdefinition or or not really? It
is.
Okay.
Chris (08:13):
It is. You know, and we've had across a lot of our
episodes recently, especiallyour fully connected episodes
like this where it's you and Idiscussing a topic. You know,
we've talked about we're kindof, you know, we've had over the
history of of the show, we'vesaw seen the evolution of AI,
and there tend to be specifictopics that get really hot. And
and right now agents are reallyhot, but there's also the notion
(08:36):
of agents applying themselvesagain, you know, in
collaboration with various otherAI technologies, whether they be
LLMs or reinforcement learningor, you know, computer vision.
And I think if you take a stepfarther into this world of
autonomy, especially modernautonomy in 2025 agents are a
big part of that, and havingmultiple agents collaborating,
(08:58):
and they're using these othermodels to get these tasks done.
So you have agents operatingwith LLMs for different
purposes. And you know, soyou're able to go and grab the
right model for the right taskand you're putting a bunch of
tasks together to go dosomething which which I might
call a mission, but I don'tnecessarily mean that in a
(09:19):
military esque way. Could be,but it could be something that
you're that is what the purposeof your company's autonomy is to
do. Know? Maybe like
Daniel (09:27):
a a goal or a objective or outcome
Chris (09:31):
That's right.
Daniel (09:31):
Could be synonyms. Maybe I'm I'm not that great at the
English language, but
Chris (09:36):
Those are all those are all loose synonyms that could
they could apply. Yeah. And soand we're seeing this, you know,
you're we're seeing theseindividual drones and robots
that are starting to do tasks inthe commercial and industrial
space. Certainly, it's been thatway in the industrial space and
warehouses and robot. I guessthe first robot I worked on was
(09:58):
2,000 that personally I wasdoing were two of them that were
autonomous in 2018.
So this is not new in thatcapacity. How it's doing it has
changed over time.
Daniel (10:07):
Yeah. Maybe it just taking a moment kind of looking
back a little bit because Iremember what what year was it
that the I forget the year ofthe Beijing Olympics when they
had the It was the first one Ihad seen where they had a kind
(10:29):
of, I guess we could call it aswarm, I don't know, You can
correct me if I'm wrong. Swarmof of drones that were sort of
in the sky in the I guess, Ithink it was the opening
ceremony. I'm I'm trying toremember, but that was the first
time I had seen that. Of course,I've seen it in other places,
you know, after that.
But, but yeah, what's what'skind of the The definition of
(10:52):
that? Yeah. Well, how would youhow would you represent the
history of kind of coming to thepoint where we're at now in
these, so I'm getting from you,there's these autonomous, so
multiple objects in the physicalworld that are autonomous or
multiple vehicles or robots orwhatever in the physical world
(11:15):
that are autonomous trying toaccomplish a goal or mission.
Now in the Beijing case, I don'tknow if those were autonomous or
not. But, yeah, help me kindaparse through maybe a little bit
of that.
Chris (11:27):
So I should say ahead of time that I don't have any
special knowledge of thatparticular configuration. So I'm
making educated guesses on that.Yeah. So I would I would suggest
that's not what I what and I'lldefine a swarm in a moment as a
follow-up. But I would not callthat a swarm.
I would call that a large numberof individual aerial platforms
(11:48):
that are operating in a in apredetermined coordinated
manner.
Daniel (11:52):
Yeah. It's like synchronized
Chris (11:53):
They're synchronized from the ground control system. So
the notion in in flyingautonomous vehicles is that you
have a ground control station,which might be like a laptop, it
might be a bigger thing inside atruck or something, but it has,
you know, comms and it has waysof communicating and and so for
shows like that, you would use,you know, you would use things
(12:16):
like GPS and they would have aeach drone would have a
three-dimensional path that it'sfollowing Mhmm. That's a little
bit different from all theothers. But when you put them
all up there at the same time,they look like a big coordinated
show, but they're not actuallycoordinated. It's the human on
the ground programming in theirpath.
It is
Daniel (12:35):
They are not determining their coordination.
Chris (12:39):
There you go, which is really important. So Okay. Yeah.
Yeah. You went right to theheart of it.
And so there's that, which isreally not intelligent at all.
It's pre programming thesedrones to do something and the
visual impact is that you havethis thing happening among a
bunch of things. But it's, it'ssort of an illusion. It's really
a bunch of individual things ontheir path. And then there's a
(13:01):
whole there's a whole continuumof how those kind of vehicles
can operate.
One kind of popular approachtoday that I that that a lot of
people would call swarming andwhich I don't, and I'll define
it right after I say this, isthe notion of like giving plays
to a group of vehicles that goout and do a task. Sort of like
(13:25):
some people the analogy will bea football play. It's go go to
this location and do this, andthere might be some
communication with the otherplatforms in your area that
you're doing something with, butI also don't consider that a
swarm. Because I think it comesback to the notion of what is
swarming. It's it's swarming isnot an autonomous thing.
(13:48):
It's a type of behaviorexhibited. And we see that in
nature. And so I will I'll givethe definition and then I'm
actually gonna go away fromtechnology for one second and
talk about what we see in mothernature that's kind of consistent
with that. So my definitionsounds a little military like,
but don't take it don't take ittoo much that way. It can be it
(14:11):
could easily be applied tocommercial or industrial.
So but I think it gets the gistof how I of how I see swarming.
Mine is, swarming occurs whennumerous independent, fully
autonomous platforms exhibithighly coordinated locomotive
behaviors in any domain, be itair, ground, sea, undersea,
space, functioning as a single,independent, logical,
(14:35):
distributed, decentralized,decisioning entity for purposes
of command, control, andcommunications with human
operators on the loop toimplement actions that can
achieve strategic, tactical, oroperational effects in the
furtherance of a mission. Littlemilitary sounding, but if you
translate those words intocommercial and industrial, they
(14:57):
still apply. You might use it toyou might choose a synonym in
some of those areas. But that isa is is a pretty high bar right
there.
And, and as we go forward, wecan talk a little bit about kind
of what in that definition, turnsomething from a collection of
autonomous vehicles into a swarmof autonomous autonomous
(15:18):
vehicles.
Sponsor (15:33):
Well, friends, if you want to build the future of
multi agent software, check outAgency. That's AGNTCY. That's an
open source collective buildingthe Internet of agents. It is a
global collaboration layer whereAI agents can discover, connect
and work across frameworks. Thatmeans better tools for you,
(15:54):
standardized agent discovery,seamless inter agent
communication and modularcomponents to scale your multi
agent workflows.
And they're teaming up with CrewAI, Langchain, Cisco and many
more dropping real code specsand services with no strings
attached. Start buildingalongside engineers who care
about high quality multi agentsoftware. Learn more at
(16:17):
agency.org. Again, that'sagntcy.org. That's agency.org.
Daniel (16:28):
Well, Chris, I I really appreciate your definition.
Maybe that's even something wecan pull out and put into the
show notes for people so thatthey can can read that, if they
if they want to. I'm actuallyreminded as we're talking about
this as an and as I'm hearingyour definition of these things,
(16:49):
you know, in the animal kingdom.Actually, I'm remembering a
movie that I watched with mywife, The Murmuring. I think it
was on Netflix and veryinteresting movie, but kind of
at the core of the movie werethese, I don't know if I'm using
the right terminology, butmurmurations of swarms of of
(17:12):
birds.
Star Lake, I'm sure
Chris (17:14):
famous for it.
Daniel (17:15):
Exactly. Yeah. So you can see in the sky, I'm sure
many people have seen thisswarms of birds that almost form
kind of patterns andsynchronizations in the sky and
and move around together. Nowaccording to your definition,
each of those birds, right? NowI guess in this sense, you know,
(17:39):
there's no connection to a,obviously a human command and
control type of scenario, butcertainly they are autonomous,
but they're accomplishingsomething together, which is
maybe a little bit mysterious inthis case, although I I'm sure
it's a topic of study.
(18:00):
But, but, yeah, I I that's whatcame to my mind is is that is
that movie and that kind ofphenomenon.
Chris (18:07):
Yeah. And that murmuration, if you will, is
literally it's called flockingin that case, you know, which is
obviously a bird term, but thatis definitely consistent with
what I would consider to be aswarm. And if you if you break
that down, mean, there's andthere are other animals we can
talk about too, they havedifferent types of swarming
behavior, but that areconsistent with that. If you
(18:29):
talk about starlings inparticular, you know, they're
beautiful when you see thosemassive, and there could be
thousands of starlings movingacross, and you see the waves
within the system, you know,within the flock as they're
doing that. And and so you kindawhat's going on there?
We have thousands of individualbeings that are communicating
(18:50):
and sensing each other. Theyhave a unified purpose. They're
all going someplace together,but there's no there's no one
master general or CEO starlingthat's going, you guys are gonna
go that way. Everyone do what Itell you to do. They are all
subscribing to the mission, ifyou will, but each one has a has
(19:11):
a a position, if you will, or aplace in that mission where they
are communicating and the waythey do their sensing of each
other and their communicationshelps them a not collide and
and, you know, fail by bythrough collisions and such.
It keeps them together, but theyalso have a way of of agreeing,
(19:32):
if you will, to go do something.They're going to to move from
one place on the earth toanother place on the earth. And
and that kinda comes down to thecrux of it, is that swarm
behavior is where you have thata bunch of fully autonomous
beings in this case, they'refunctioning in that murmuration
(19:53):
as a single independent logicaldistributed decentralized
decisioning entity. It's a longmouthful that I use because all
of those have a part in thatdescribing it, but they're all
working together in that way.
Daniel (20:06):
Do you wanna pick apart maybe each of those words? So
just like why why use thosemultiple words kinda
distributed, decentralized? Somepeople might kind of not know
why each of those is isimportant.
Chris (20:20):
Sure. So so single implies that the joint actions
together about how they'recommunicating and sensing give a
rise to one generalized process.
Daniel (20:34):
Kind of an emergent or combined
Chris (20:36):
process. That's a great way of putting it. And that's
kind of that that emergent, youknow, we said mission or goal
kind of thing is coming out ofthat. And that process is
independent of any one bird. Andit's and it's independent of any
controlling authority thatsaying from afar, y'all shall go
do this thing, you know.
And so you have thatindependence, you have that
(20:59):
single emergent characteristic,logical meaning, it's also kind
of, it's kind of think about,it's arising a an emergent
intelligent notion or aprocessing notion that's all
happening by the unity. It isdistributed across the entire
flock or murmuration. It isdecentralized. There's no one
(21:22):
special bird or group of specialbirds that are the birds that
are in charge of everybody elseand you guys have to do what we
say. And finally, it's adecisioning entity that by by
this emerging thing happening,choices are made collectively
for the whole thing.
And yet, no one bird is saying,everyone's going to go do this
(21:42):
thing. So it's all these kindsof characteristics about that
emergent, I may add the wordemergent into the definition
that I have there, because Ithink that's really a great way
of describing that. But it'sthat unified collection to do
this. Now, that's a far cry fromwhat we're currently doing in
the autonomy space at large. I'mnot saying that doesn't exist in
(22:06):
little pockets, academic pocketsand other places, labs, whether
they be government labs oruniversity labs and stuff like
that.
But you don't see widespreaddeployment of drones or robots
that are acting in this manneras we sit here recording today.
Daniel (22:24):
Okay. That that makes sense. Yeah. Yeah. Part of my
what I'm thinking and myquestioning in my mind, which
maybe you're going to get to asyou kind of get get through some
of these things is, obviously,we have a kind of inspiration or
a model there from the theanimal kingdom.
In the case of, let's say,robots in a warehouse or a
(22:49):
manufacturing facility orautonomous vehicles, we have in
probably things that people arefamiliar with, the idea of self
driving things or autonomy. Solike self like the Waymo cars or
that sort of thing. And there'scertainly, obviously, AI at play
(23:09):
there in terms of computervision and maybe reinforcement
learning and decision making. Soassuming that people might know
at least or be exposed to that,in terms of the connection to AI
specifically, what about thiskind of swarming or, you know,
(23:33):
robots or these entities workingtogether to do a task? What is
necessary for AI to do in thatswarming scenario that's not
that's not necessary in just asingle, like, self driving car,
like a Waymo car or somethinglike that?
Chris (23:50):
That's a great question. There's a lot of nuance there.
So so the self driving car ishaving to navigate an
environment in which it's bothsensing and to some degree
communicating potentially,although that is evolving over
time, with with other actors inthat environment around it. And
there is emergent behavior onwhat to do. There's kind of the
(24:14):
classic cases of, you know, kidsplaying on the sidewalk and the
ball bounces into the street,know, and and reacting to
various unexpected events likethat.
But at the end of the day, thatvehicle is still, which may be
fully autonomous, is still onlyhaving to decision on its own
behalf. And so it has an onboardinference system, the onboard
(24:38):
computers, the onboard models,or potentially that in
combination with some remotethat's doing over some sort of
connection, radio connection orcellular connection, but it's
still decisioning for itself.The difference according to my
definition of a swarm is thatthe emergent quality of of the
decisioning is arising fromparticipation in many of the
(25:04):
platforms that are participatingin that unified mission or
activity. But it has to decidefor all of them and it may have
to decide on platform A, we'relooking at something, we're
evaluating it, we're sensing it,we're processing based on
reinforcement learning and ouragents are then saying platform
a, you go do this activity.Platform b, you go do that one.
(25:26):
Platform c, you go do that one.And those are all slightly
different activities as thoseplatforms collaboratively work
together based on the sensingand inferencing that's happened
across the swarm to get anactivity accomplished. Does that
clarify a little bit?
Daniel (25:45):
Yeah. That that helps. And I'm trying maybe this is the
decentralized part of whatyou're talking about, but part
of maybe what is a challenge inmy mind to think about is how
such a thing would happenwithout a central kinda, so
(26:05):
like, so to say like, oh, let'sjust take a very simple maybe
it's simple, you know, in mymind it seems complex as well,
but we have 20 robots in amanufacturing facility trying to
do something. In one model, Icould see where those, each of
those robots detects certainthings on a number of sensors,
(26:30):
maybe cameras or temperaturesensors or force sensors or
something like that. All of thatis communicated back to a
central processing system.
And it's really the singleentity, the kind of master brain
that makes the decision andcommunicates out the next step
to all the robots. But in adecentralized way, you talked
(26:53):
about how there's not this kindof central there may be a
central command where maybethere's oversight or something,
but things somehow happen in adecentralized way. That's what
I'm trying to connect in my inmy brain maybe.
Chris (27:07):
No. That's that's a great nuance that you're pointing out
there. So in the warehouse,that's not really that that's
definitely a centralizedenvironment, and you don't even
have to have onboard compute forthat because your robots can
have a highly reliable system ofcommunication over WiFi, you
know, with a master serverthat's there and you're in the
(27:28):
distances involved or closeenough to where you can have
them interact with each otherwithout having to do that. And
according to the definition ofthe swarm that we're using here,
that would not be a swarm, eventhough you may have many
platforms working
Daniel (27:42):
Multiple devices.
Chris (27:43):
Collaboratively, they're being controlled by by a local
centralized agent. And inswarming, generally, according
to this definition, you'rereally looking at an environment
where that's not possible.You're looking, you know, what
we would traditionally call edgeor or, you know, far edge is
another term that what peoplehave used. Basically, where you
(28:04):
can't count on either local orcloud environments to provide
the compute. And so you have tohave the compute on board,
including all the activitiesthat will arise from that
compute.
Daniel (28:16):
Yeah. I'm almost imagining moving still kind of
in my mind living in theindustrial world if I think more
to like oil and gas and likecertain of the environments that
are part of part of working inthat industry, whether it be
deep under the ocean or in verydisconnected remote places where
(28:42):
you might not even want to exwant to or even could expose
humans to those environments.
Chris (28:49):
That's right.
Daniel (28:50):
But they're also very hard in terms of connectivity
and and that sort of thing. Youmight want to accomplish tasks
in in that sort of environment.Am I getting to the right kind
of scenarios?
Chris (29:01):
Okay. Like, one good one that is commonly used is the
notion of disaster recovery,where you might go into an area
that is geographic, you know, itmay be geographically remote or
may not be, but either way theinfrastructure has likely things
have happened with theinfrastructure to where things
have been torn down, you don'thave cell towers, things like
(29:23):
that, and you don't have thelevel of connectivity necessary
for centralized control. And soif you're thinking about
disaster recovery in a swarmcontext, you where you do not
have that infrastructure inplace to guide those, then you
could put drones or robots onthe ground that have their own
(29:45):
inference capability. And ifyou're approaching it with that
decisioning entity that isdecentralized that I've
described then that could occurwhere they're actually working
collaboratively to save lives.You know if there's buildings in
rubble, know one robot that hasthat ability as maybe pulling
(30:06):
rubble off because it has thatcapability while others that
specialize in getting downbetween the rubble and they're
smaller, might be in otherwords, not all robots may be the
same.
You might have a heterogeneousmix of robots with different
capabilities, but they're ableto dynamically go, here's a rock
and it needs to be moved whileyou go under the rock, you other
(30:27):
platform go under the rocklooking for a survivor there.
And so you can have a collectionof robots that each has an
optimal function. Yeah. Wherethey're working together to do
something and ultimately save alife and you don't have, you
know, maybe you've had hurricanecome through or tornadoes and
you don't have the normaldigital infrastructure that
(30:50):
we're all so used to today.
Daniel (30:52):
Well well, Chris, that that is really helpful to break
down maybe some of the use casesor scenarios that maybe compare
and contrast multiple agents orrobots or systems that are
working in an environment thatmaybe are in a swarm or are not
(31:13):
in a swarm. Maybe part of sothat's helpful on the use case
front. My mind is still kind ofwondering on this side of the AI
side, what unique AI problemsare unique to the swarm
environment that aren'tencountered elsewhere? And how
(31:36):
do those map to kind of maybeopen challenges that are that
are open or kind of maybe knownmodels that are able to handle
certain of these things?
Chris (31:45):
Yeah. So there's I mean, I think you just kind of alluded
to the complexity involved. AndI think the reason we haven't
had swarms before now and thatthey are still something that
that is emerging within what isbeing developed is because not
all of the technologies andmodels that you would need have
(32:05):
been mature enough up until thispoint in in time to be able to
do that. So there's a lot ofpieces. And some of that is on
the AI side and some of that ison is operating in a physical
environment.
And so on the AI side, you know,over the years, we've talked
about these differentarchitectures, you know,
(32:26):
different models with differentpurposes. And swarming ends up
taking quite a few of those andmaking them available to agents
so that you can within acomplicated physical
environment, different agents onplatform are working with each
other to accomplish the onplatform parts, and then be able
(32:47):
to communicate and coordinatethat with other platforms that
also each are multi agent multimodel configurations. And so and
then being able to do that.
Daniel (32:58):
Swarm of swarms.
Chris (32:59):
Yeah, sort of like if you think of all the various
technologies that go into just asingle platform able to do that.
We're only now really gettingyou know, in the large getting
to where these kinds of thingsare possible. And where the
compute is powerful enough foredge devices and that the models
are being reduced. We've talkeda lot over the last year or two
(33:21):
about how models are gettingsmaller and you get a hugging
face and, you know, the vastmajority of models there are
very small models that areuseful in a lot of specific
tasks. And so you have to matchup small models that can run on
edge inference or or even CPUsto do various tasks for specific
agents and coordinate that withsensors and comms.
(33:43):
Yeah. To be able to accomplishstuff. So it's quite
complicated. And that kind ofand I use this kind of in air
quotes, that kind ofminiaturization of the
technology. I don't meanphysical miniaturization, but
kind of getting everything towork and run on a remote edge
device that is that is likelybattery powered is is a bit of a
(34:05):
challenge.
So I think, as we look forwardright now, there's, you know,
we're, we're really at ainflection point in history in
terms of being able to do this.
Daniel (34:14):
Yeah. And what just did a very practical level, I'm sure
there's a great diversity here,but in some of what you're
talking about, there's a uniquecommunication element and goal
setting element of thesesystems. And if I look at like
AI models that are running onany particular platform within
(34:36):
the swarm, what, at a very highlevel, what kinds of models and
tasks need to be accomplishedthat are associated with the
swarming versus just likesensing and that sort of thing?
Are these models that are, youknow, small SVMs or something
(34:58):
that are making a kind of binarydecision or gradient boosting
machines or neural networks orgen AI models? Like what is the
diversity you see and what arethe kinds of tasks you just, by
way of example, a couple of themthat might need to be done?
Chris (35:14):
Yeah. I I it's a mixture of different models, some of
which can be more traditional AImodels like what we most often
talk about here on the show.Some of them are more of the
classical data science modelsbecause, you know, we've talked
about this a number of timeswhere you don't need to go to a
bigger, harder, you know,expensive model to use if a
smaller model will work, butit's very task specific and then
(35:37):
and it's bound within thesoftware systems of each
platform but also the softwaresystems which govern sensing and
comms. And then so you need theability for different nodes, if
you will, of what it you know,different platforms to maybe on
behalf of the swarm to take onsome of the computation because
you can't distribute a modelinference across all of the
(36:00):
things. But what you can do issay, if you have a swarm of a
dozen, I'm just obviously makingthis up off the top of my head,
have three of the platformsrunning inference on a
particular model for aparticular task while three
others do another thing and thenyou can you can have things like
election algorithms which decidewhich one you're gonna take.
(36:22):
And so you can you can haveevaluation of what who's getting
the best sensing information andwho has the best comms reach for
the entire swarm to ensure thatdata gets around. So you can
have some redundancy in thecomputation and all those things
and then by election pick andelection's just one mechanism.
That's one common algorithmicmechanism that that one can
(36:45):
apply. But by election withinthe swarm, choose who is
issuing, you know, the resultsof the various types of
inference that the agents areputting together and deciding
upon a task. So that you canthen go and assign specific
platforms to do different partsof that task, where not every
platform is doing the samething, but they're all
(37:05):
contributing to the larger goal.
Daniel (37:07):
Gotcha. Yeah. That's very helpful. Now I'm sure
there's some people out therethat are maybe excited by this,
disturbed by this topic, maybe.Definitely.
I mean, one sense, just again,thinking about things outside of
the robotic world, we talkedabout the birds, which are
(37:29):
beautiful and you see thepatterns which they form. I also
think about like in the humancontext, there's interesting
maybe swarm behaviors that aresomewhat disturbing. Like if you
think about a mob mentality, youoften hear about people, you
(37:49):
know, maybe a a mob of peopleall accomplish a very disturbing
thing, like a destructive thing.And then afterwards, you know,
individuals that were in thatswarm say, I don't know what,
you know, the individuals mightnot ever have done what was done
by the mob, right? But becauseof this emergent behavior,
(38:14):
things happen that maybe areoutside of the bounds of any
individual's, you know, moralcompass or that sort of thing.
So if we take this then to therobotic or drone or industrial
side, some people might say,well, how do we push one
direction and not the otherdirection? And what is the
(38:34):
oversight related to this?
Chris (38:37):
So it's a great question. I'm glad I'm glad you brought it
up in my enthusiasm for thetechnology. It's very easy to
lose sight of this. And we had avery recent show where we talked
a little bit about, know, theanthropic, I would refer people
back to that. And the fact thatyou had, you know, models that
were coming out with what wedeem to be bad outcomes.
And that can certainly applyhere. And so you have to have
(39:00):
some form of guardrails. And sothere's two, I want to start
with, there's two phrases, oneof which I mentioned before. I
said human operators on theloop. And human operators on the
loop versus human operators inthe loop are two distinct
things.
Right now in the in the militaryspace, based on current
(39:22):
regulations here in The US andstuff, you you most often are
talking about in the loopinteractions between humans and
stuff. But as you go forward,and this is not a military
specific thing, this issomething that you're gonna see
in commercial and industrial aswell, as you start talking about
swarm capabilities with emergentproperties happening, then the
(39:47):
challenge is that there aresometimes you want a human to be
able to step in the loopdirectly into that processing
and say yes, no or make aselection or something like
that. But there are other timeswhen you're talking about a
larger swarm where you need ahuman who where the it may be
there may be too much going onand there may be too many of
(40:08):
these platforms flying around oron the ground for one human to
track everything at every momentand interact but you can have a
human on the loop, can say forthe overall goal or for the
thing the swarm decides approveor disapprove, you know, if
there an emergent behavior comesup, and the swarm makes a
decision based on what it'sseeing in real time, then you
(40:31):
can say, No, that's not going towork.
And that's what a human on theloop does is you can it gives
you that guardrail of saying,that's not the emergent behavior
I want, you know, it's time togo back or maybe do a full
recall and shut down. So I thinkif you really want to learn more
on these kinds of things you'restarting to see I it's, I'm not
(40:52):
sure that I can think of aspecific swarm. I'll have to do
some research and see if there'sany specific swarming classes. A
lot of the kind of drone androbot systems that you can get
out there in open source. Forinstance, there's ROS and ROS
two, which is the roboticoperating system, first and
(41:14):
second version.
ROS two takes a slightlydifferent approach to ROS one,
so there are some people on eachversion. I know ROS two has some
swarming capabilities in thereto build on. It doesn't
instantly give you swarmcapability but it has some tools
in there that you can startbuilding on and that might be a
good place to start to learnabout the topic. But in general,
(41:38):
kind of staying tuned intolearning sources, including this
podcast, where you can learnabout multi agent environments
and as we increasingly aretalking about physical world
deployments of AI technologies,those are all good places to to
consume because this is a verycutting edge area to to be
focused on.
Daniel (41:58):
Yeah. It seems like a lot of a lot of things are
developing, and I personally amvery excited about some of the
things happening in robotics.Obviously, we've maybe some of
us, the more we see chatinterfaces, it's like we just
wanna see some other applicationof of AI technology, which that
(42:21):
is only that's such a smallsegment of what is possible with
AI, even Gen AI, there's so manymore ways that that will, I
think, become embedded in ourphysical spaces. If you just
wanna search for kind of thisidea of physical AI, I know
that's a big topic of kind of AIembedded in our physical spaces,
(42:46):
maybe not through chatinterfaces, but through other
interfaces. And one thing thatI've really enjoyed seeing is
the stuff coming out of, PollenRobotics, p o l l e n.
That's a company that I believeif if I'm not mistaken was
acquired by Hugging Face andthey're releasing a lot of well,
(43:10):
I don't know a lot. They'rereleasing robotic systems that
are more geared towardsexperimentation, open source
development, integration of openmodels, integration of apps and
exchanging of those apps evenwithin Hugging Face spaces but
for actual physical systems.Actually, Prediction Guard, our
(43:32):
company, we got one of the orordered one of these, I think
it's Ricci Mini, robots. Lookingforward to playing around with
that in our office. That shouldbe really fun.
So I think also there's anincreasing number of ways that
people can access even physicalsystems or robots in a much more
(43:57):
accessible way than before wheremaybe everything was coming out
of whatever it was, BostonDynamics or or wherever very,
very expensive millions ofdollars systems here. There's
ways to access these kind ofsystems, whether it's drones or
robots or that sort of thing inin a in a much more accessible
(44:18):
way, which is reallyencouraging.
Chris (44:19):
It's not expensive anymore. So in the maker space,
there is a lot of resources. Alot of it is based on stuff you
may already know, like Arduinoor Raspberry Pi, and you know,
adding a Jetson in their carrierboards where you basically get
some basic stuff. So we're gonnasee more and more of this going
forward. And as people reallykind of get even some of the
(44:41):
more advanced topics figuredout, It's gonna be in all of our
lives in the years ahead.
So we're gonna see a rapidintroduction into daily life of
these things. And so if thistopic interests you, definitely,
I would encourage you just as Ido, is to go out and jump into
the maker space and and for verylittle money, you can have some
(45:03):
pretty interesting experiencesbuilding some of these things.
Daniel (45:06):
Yeah. And I it's occurring to me that after this,
we'll we'll put up, maybe a awebinar, that where we could
talk about these things a littlebit more, maybe even
demonstrating some things withwith robotics. So don't forget
to check out practicalai.fm/webinars for, upcoming times
(45:28):
that that will be live wherediscussions can happen. And,
yeah, just really appreciate youhelping us deep dive into these
subjects, Chris. It's been it'sbeen really good.
Chris (45:38):
It's a fun topic. Happy to do it. Thanks a lot, Daniel.
Daniel (45:41):
Alright. See you soon.
Chris (45:42):
See you later.
Jerod (45:49):
Alright. That's our show for this week. If you haven't
checked out our website, head topracticalai.fm and be sure to
connect with us on LinkedIn, X,or Blue Sky. You'll see us
posting insights related to thelatest AI developments, and we
would love for you to join theconversation. Thanks to our
partner Prediction Guard forproviding operational support
for the show.
(46:09):
Check them out atpredictionguard.com. Also,
thanks to Breakmaster Cylinderfor the Beats, and to you for
listening. That's all for now,but you'll hear from us again
next week.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Special Summer Offer: Exclusively on Apple Podcasts, try our Dateline Premium subscription completely free for one month! With Dateline Premium, you get every episode ad-free plus exclusive bonus content.
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy, Jess Hilarious, And Charlamagne Tha God!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.