October 24, 2025 • 48 mins
In this fully connected episode, Daniel and Chris explore the emerging concept of tiny recursive networks introduced by Samsung AI, contrasting them with large transformer based models. They explore how these small models tackle reasoning tasks with fewer parameters, less data, and iterative refinement, matching the giants on specific problems. They also discuss the ethical challenges of emotional manipulation in chatbots.
Featuring:
Links:
- Less is More: Recursive Reasoning with Tiny Networks
- Researchers detail 6 ways chatbots seek to prolong ‘emotionally sensitive events’
Sponsors:
- Outshift by Cisco - The open source collective building the Internet of Agents. Backed by Outshift by Cisco, AGNTCY gives developers the tools to build and deploy multi-agent software at scale. Identity, communication protocols, and modular workflows—all in one global collaboration layer. Start building at AGNTCY.org.
- Fabi.ai - The all-in-one data analysis platform for modern teams. From ad hoc queries to advanced analytics, Fabi lets you explore data wherever it lives—spreadsheets, Postgres, Snowflake, Airtable and more. Built-in Python and AI assistance help you move fast, then publish interactive dashboards or automate insights delivered straight to Slack, email, spreadsheets or wherever you need to share it. Learn more and get started for free at fabi.ai
- Miro – The innovation workspace for the age of AI. Built for modern teams, Miro helps you turn unstructured ideas into structured outcomes—fast. Diagramming, product design, and AI-powered collaboration, all in one shared space. Start building at miro.com
Upcoming Events:
- Join us at the Midwest AI Summit on November 13 in Indianapolis to hear world-class speakers share how they’ve scaled AI solutions. Don’t miss the AI Engineering Lounge, where you can sit down with experts for hands-on guidance. Reserve your spot today!
- Register for upcoming webinars here!
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Jeord (00:04):
Welcome to the Practical AI podcast, where we break down
the real world applications ofartificial intelligence and how
it's shaping the way we live,work, and create. Our goal is to
help make AI technologypractical, productive, and
accessible to everyone. Whetheryou're a developer, business
leader, or just curious aboutthe tech behind the buzz, you're
(00:24):
in the right place. Be sure toconnect with us on LinkedIn, X,
or Blue Sky to stay up to datewith episode drops, behind the
scenes content, and AI insights.You can learn more at
practicalai.fm.
Now onto the show.
Sponsor (00:39):
Well, friends, it is time to let go of the old way of
exploring your data. It'sholding you back. Well, what
exactly is the old way? Well,I'm here with Mark DePuy,
cofounder and CEO of Fabi, acollaborative analytics platform
designed to help big explorerslike yourself. So, Mark, tell me
about this old way.
So the old way, Adam, if you'rea a product manager or a founder
(01:02):
and you're trying to getinsights from your data, you're
you're wrestling with yourPostgres instance or Snowflake
or your spreadsheets, or if youare and you don't maybe even
have the support of a dataanalyst or data scientist to
help you with that work. Or ifyou are, for example, a data
scientist or engineer oranalyst, you're wrestling with a
bunch of different tools, localJupyter Notebooks, Google Colab,
(01:23):
or even your legacy BI to try tobuild these dashboards that
someone may or may not go andlook at. And in this new way
that we're building at ABBYY, weare creating this all in one
environment where productmanagers and founders can very
quickly go and explore dataregardless of where it is. So it
can be
in a spreadsheet, it can
be in Airtable, it can be inPostgres, Snowflake. Really easy
(01:44):
to do everything from an ad hocanalysis to much more advanced
analysis if, again, you're moreexperienced. So with Python
built in right there and our AIassistant, you can move very
quickly through advancedanalysis. And the really cool
part is that you can go from adhoc analysis and data science to
publishing these as interactivedata apps and dashboards, or
(02:06):
better yet, delivering insightsas automated workflows to meet
your stakeholders where they arein, say, Slack or email or or
spreadsheet. So, you know, ifthis is something that you're
experiencing, if you're afounder or product manager
trying to get more from yourdata or for your data team today
and you're just underwater andfeel like you're wrestling with
your legacy, you know, BI toolsand and notebooks, come check
(02:28):
out the new way and come try outFabi.
There you go. Well, friends, ifyou're trying to get more
insights from your data, stopresting with it. Start exploring
it the new way with Fabi. Learnmore and get started for free at
fabi.ai. That's fabi.ai.
Again, fabi.ai.
Daniel (03:00):
Welcome to another fully connected episode of the
Practical AI podcast. This isDaniel Wightnack. I am CEO at
Prediction Guard, and I'm joinedas always by Chris Benson, who
is a principal AI researchengineer at Lockheed Martin. And
in these episodes where it'sjust Chris and I, we like to
(03:21):
dive into certain topics thatare trending in AI news and, you
know, help both us and youhopefully, level up your AI and
machine learning game. How youdoing, Chris?
It's it's good to be back to oneof these episodes with just the
two of us and maybe explore atopic that we can both learn
(03:42):
about.
Chris (03:42):
Absolutely. Yeah. I I love these episodes of us just
kinda bantering whatever wehappen to wanna do. Love the
guest episodes too, but thereit's kind of a different beast
in that way of exploring whatsome person organizations doing.
And there's so many cool thingsthat we can just dive into.
And I think we have a few thisweek.
Daniel (04:03):
Yeah, yeah, at least a first very, very tiny topic to
to discuss, I was actually Oneof our engineers brought this up
to me. I forget it was earlierin the week, but this idea of
tiny recursive networks. Youknow, all the time we're talking
(04:23):
about transformer based LLMs onthe show and generative AI. And
I kind of personally always lovegetting back to a little bit of
cool data science and researchstuff just to see where the
industry is headed because thisis a kind of different animal
(04:43):
that we'll be talking about,this tiny recursive networks or
models, and operates differentlythan kind of the hype GenAI
models of today. And it doesmake me think, and I don't know
if this is something you've beenthinking about, Chris, but kind
of we're all the time talkingabout GenAI, we're talking about
LLMs, now we're talking aboutagents, all of those being
(05:06):
driven by these transformerbased LLMs.
And certainly people havetalked, prominent people have
talked about the fact that weneed to get beyond transformer
based LLMs. And of coursethere's many companies that are
just centered around these typesof models. So any thoughts on
(05:27):
that of your own predictions orthoughts of when we're kind of
headed to the next phase of whatmodels will look like beyond
just transformer based LLMs.
Chris (05:42):
Yeah, I mean, I think I'm gonna sound like a broken record
on this because it's not new forme. And that is, you know, I
agree with you. We're always,you know, the hot things that
the media tends to follow ingeneral are the big LLMs. But
you know, because I guess it's,you know, it's the next giant
thing sexy, you know, to talkabout, but like, the world is,
(06:05):
you know, all these technologiesare moving from the cloud out
into the world, into physicalAI, you know, and robotics and
all sorts of ways that weinteract with, you know, not
just LLMs, but all sorts ofmodels out there in the world
where, you know, every one ofour lives is touched in so many
different ways. And, and that'sexactly, you know, as we were
(06:26):
diving into this, here, thistopic with the tiny recursive
networks, That's, that's what itseems like to me.
I'm looking forward to talkingabout that. Because as I
mentioned to you right beforethe show started, you could see
these popping up everywhere,like in all sorts of different
use cases.
Daniel (06:43):
Yeah. And I I just to set the stage why this is maybe
intriguing, there was a paperthat that came out less is more
recursive reasoning with tinynetworks. This came out from
Samsung's AI lab in Montreal,specifically there's an author,
Alexia, on this article, whichif you're out there listening,
(07:07):
we'd love to have you on theshow. Please come join us and
talk more about this. Hopefullywe won't butcher this work too
bad as we talk about it.
You'll hear Chris and I kind oflearning as we go on this
episode, as we talk back andforth about what this exactly
is. But this is, you know, thepaper is less is more cursive
(07:27):
reasoning with tiny networks.And I think the major thing
that's interesting here isthere's a model that they talk
about that has only 7,000,000parameters, which is tiny. Yeah,
yeah. So like, just wanna sortof let that sink in.
(07:48):
So I didn't say 7,000,000,000parameters, 7,000,000 parameters
with an M, which yeah, Chris, ifas we've basically gone about
this trend, I mean, actually a7,000,000,000 parameter model
now is quite small.
Chris (08:05):
Yes. In a sense. 27,000,000 compared to what we
traditionally call very small atthe 7,000,000,000 level. I mean,
this is that, you know, thereit's using the word tiny for a
reason. But, yeah, it does youknow, when you think of millions
as being as being almostnothing, it's an interesting
context shift there.
Daniel (08:25):
Yeah, yeah. So I actually love this because I
love the idea that we could moveinto a phase where we're dealing
with models that are very small,can run on commodity hardware or
at least be smaller in size.And, you know, they may run for
longer periods of time or theremay need to be optimization
(08:46):
around how they run recursively.We'll get into that recursive
bit, but certainly a smallmodel. But it was shown to kind
of have, let's say comparable oron par performance with some of
the big guys.
So we're talking like DeepSeqR1, Gemini 2.5 Pro. These are
billions and billions ofparameters models, very huge
(09:09):
transformer based LMs, thesekind of reasoning models that
we've talked about on the show.The center of this work is
really kind of related to thesereasoning tasks. Now, right out
of the bat, I think it would beworth saying that these, it's
not like this tiny recursivenetwork is a general purpose
(09:32):
model that can do whatever youwant it to do. It was trained
for a very kind of small numberof tasks, but these were
reasoning tasks that some ofthese other models like a
DeepSeekAR1 or somethingsometimes has quite a bit of
issue with.
So solving like math or Sudokutype of puzzles. And I know
(09:56):
Chris, we had talked to, well, Idon't know if you wanna refresh
some of your Sudoku experiments,but
Chris (10:03):
Yeah, what you're referring to is some episodes
back, I'd have to look up andfigure out where it was. I was
playing with GPT four at thetime on Sudoku and it was just
doing a terrible job on Sudokuand giving it you know, a lot a
lot of just really bad output interms of I mean, honestly,
(10:25):
crappy answers. And so that wasthe really the first thing I
noticed on this thing was thefact that because they call out
Sudoku as being one of thethings is that these tiny models
being trained for very specifictasks, and that it could
potentially outperform theselarge models on specific things
like Sudoku being one exampleand and others as well. But I
(10:48):
think in a in a slightly largersense, this is much more real
world applicable the way I seeit in that as the as we have
models spreading across theworld for a lots of different
tasks, this is perfect for that.Yeah, it's not one model to rule
them all.
In most real life situations,it's really a collection of very
specific models that each does atask very, very well and is
(11:11):
efficient at that. And I thinkthis is a great example of that.
Daniel (11:16):
Well put in and it's probably worth reminding
ourselves about, like as wehighlight the differences with
this model, reminding ourselvesabout transformers. So, you
know, as we've gone through theprocess from deep learning to
recurrent neural networks andtransformer based self attention
(11:38):
networks. If you imagine what wehave with these big LLMs, what
happens is you put in a sequenceof tokens, which are represented
by numbers. You put in asequence of tokens that are
represented by numbers, thesetokens being kind of words or
sub words. And all of thosetokens are processed in a
(12:01):
forward pass through a giant setof, if you wanna think about it,
like sub functions, which addand multiply and combine those
numbers through a very vastnetwork of functions to generate
many different probabilities ofkind of next words coming out,
(12:25):
which allow you to predict kindof a completion of words or a a
set of reasoning or a set ofthinking or a solution to a
problem, right?
This is how these networks work.And I would recommend people,
we've had Jay Alomar on the showbefore. He has some great kind
of the illustrated transformerblog posts. So Jay, shout out to
(12:49):
you. Thanks for doing that.
I would take a look at thoseblog posts. They do a great job
at explaining this more visuallyfor those where that would be
helpful. But the main kind ofthing here that I'm saying is in
the models that we're using now,basically it's a giant function,
if you want to think about itthat way. It's a data
transformation. You putsomething in one end, it
(13:12):
processes through one way,through the function and
produces a result.
Now you may run that functionmultiple times to produce
multiple words out the otherend, which is what happens when
you stream output into like achat interface. But ultimately
each time the model runs, it's asingle run through the model
(13:34):
input, it's transformed to someoutput. That is not recursive as
we would say with these models.And if you want to think about
it, it requires very, very largemodels because what you're
modeling is a very complicateddata transformation. So for you
(13:58):
to put in some text related to amath problem and predict the
right wording of a solution outthe other end, that's actually a
very non trivial datatransformation, right?
Which means you have to have avery large function to kind of
fit or model that datatransformation, which is why
(14:18):
these models have become solarge. So now with this tiny
recursive setup, what'shappening is you're not just
looking at the model as a singleforward pass data
transformation, but youintroduce the idea of recursion,
(14:39):
which means you sort of outputfrom the model and that output
becomes the input for the samemodel, which creates this kind
of circle or recursion, which iskind of interesting. So you're
essentially trading what wouldbe a very large function to
(15:02):
model that data transformationfor many, many kind of recursive
runs of a single, very smallmodel. That's maybe a simplified
way to put it. And we can get alittle bit more into the model
itself here in a second.
Chris (15:19):
So I have a question for you on this is one of the things
when I was mentioning the27,000,000 earlier, was talking
about the hierarchical reasoningmodel, which is a previous model
put out there versus the tinyones, which have the five to
7,000,000 parameters. Can youtalk a little bit about like,
are they completely differentthings? Is the tiny an outshoot
(15:42):
of the hierarchical? How youcompare those two?
Daniel (15:45):
Yeah, good point. So just like everything we talk
about on this show, sometimes itdoes seem like things pop out of
thin air like tiny recursivemodels now. But in reality,
there is a buildup ofincremental research that leads
to new technology or newfindings. One of the things in
(16:07):
that kind of lead up to thesetiny recursive models was a
previous work aroundhierarchical reasoning models.
And these also were smaller,like you're saying, 27,000,000
parameters is still very smallin today's standards for models
at least.
But these hierarchical modelsactually use two very small
(16:33):
transformer networks, fourlayers each, and they recursed
between each other. So I don'thave the full details of that
and wouldn't be able to explainit if I did probably, But that's
the main idea is thesehierarchical reasoning models
have these two models thatrequired two networks, two
(16:55):
forward passes per step andcreated some, I guess,
complications because of that.So this introduction by Alexia
and the Samsung team here is asingle network. So it uses one
tiny network with two layers. Soa single tiny network with two
(17:18):
layers that roughly has kind offive to 7,000,000 parameters and
it operates kind of in thisrecursive refinement.
So it recurses on itself, if youwill. And the hierarchical
reasoning model with the27,000,000 parameters, for
example, on the Sudoku extremebenchmark scored a 55%. I don't
(17:42):
know the exact kind of way thatthat's scored, but just by kind
of comparison, the tinyrecursive network, which is even
tinier, when training on athousand examples was able to
achieve 87%, accuracy on SudokuExtreme.
Sponsor (18:05):
Well, you don't have to be an AI expert to build
something great with it. Thereality is AI is here. And for a
lot of teams, that bringsuncertainty. And our friends at
Miro recently surveyed over8,000 knowledge workers. And
while 76% believe AI can improvetheir role, most, more than
half, still aren't sure when touse it.
(18:28):
That is the exact gap that Mirois filling. And I've been using
Miro from mapping out episodeideas to building out an entire
new thesis. It's become one ofthe things I use to build out a
creative engine. And now withMiro AI built in, it's even
faster. We've turned brainstormsinto structured plans,
screenshots into wireframes, andsticky notes chaos into clarity,
(18:52):
all on the same canvas.
Now you don't have to masterprompts or add one more AI tool
to your stack. The work you'realready doing is the prompt. You
can help your teams get greatdone with Miro. Check out
miro.com and find out how. Thatis Miro dot com, mir0.com.
Daniel (19:15):
So Chris, just to kinda drive, I guess, the point home
here with these tiny recursivenetworks, you have this single
tiny network and you areessentially replacing, if you
wanna imagine a big kind ofpipeline of processing, which is
(19:35):
what these big LLMs are, And yougo one pass through the whole
pipeline of processing. Here,the pipeline is smaller. You've
got less layers, lessparameters, but you replace that
kind of depth with iteration. Soinstead of stacking those
transformer blocks, you repeatthe network over and over
(19:58):
essentially to kind of refineits reasoning state or the
solution guess, right? And sothis iterative refinement, one
of the things also is it kind ofhelps avoid overfitting on small
datasets, which to your point,Chris, earlier about real world
(20:20):
business cases, often thereality is that you kind of
don't have You have scarcity ofdata for very many problems.
You don't often have a reallynice kind of large, millions and
millions of things to train on.Right?
Chris (20:37):
You know, and I think that's one of the most
fascinating things about this isin the paper, they talk about
the fact that, they're achievingthis higher accuracy on hard
puzzle benchmarks, whiletraining on only approximately
1,000 examples. And, and thatwhen you think about, you know,
the challenge of having a greatdata set in the more traditional
(21:01):
context that we've been talkingabout, and that becomes such a
challenge for many people andorganizations to do that. But
it's a lot easier to get 1,000,know, 1,000 examples together.
And it puts, you know, not onlyfrom the computational side, but
also from the the data set sideputs this much more in reach for
(21:22):
a lot of problems that peoplemay have, where they do want to
solve a narrow, a narrow concernwith high accuracy. And so I see
this as it's kind of the everyperson's way of modeling in
terms of tackling things goingforward without a lot of
resources and a lot of maybe nota lot of time to put things
(21:42):
together, you could probably doit pretty quickly.
Daniel (21:44):
Yeah, yeah, exactly. And I guess just to kind of put some
of the boundaries that arecurrently around these recursive
models, One of the things I wastrying to parse through as I
looked at this was, well, whatis the setup now? How general is
(22:04):
the output? How kind of generalcan the output or the input and
the output be? And part of thetrick here, you know, it's not a
trick, but part of the setup isthat these tiny recursive
networks, they don't take a kindof unstructured, natural
(22:28):
language text input.
They take some structuredrepresentation of a whole
problem at once. So you canthink of a puzzle grid in Sudoku
or a math word problem turnedinto structured features or a
reasoning question encoded bynumbers or symbols or logic,
(22:51):
something like that. So insteadof feeding in the input kind of
word by word, like a chatbot,you're giving a situation to the
model. It's kind of a one shotsituation, which is then turned
into of course embeddingsinternally because, you know,
computers work on series ofnumbers, right? That's the only
(23:14):
thing a computer can process isnumbers, right?
But those numbers, kind of thatembedding represents a kind of
one shot of a problem, which isinteresting because, you know,
it almost seems flashback likewe're kind of coming full circle
to reasoning problems, but in amore data science y way than a
(23:36):
generative AI way, which is kindof refreshing and cool.
Chris (23:41):
That's very much what I was about to say in the sense of
this feels a lot more like kindof traditional pro like the way
that you put a problem togetherand more of a traditional
software development way, whereyou you know, you'll create some
structures and you'll pass themin. And when we got to Gen AI,
and then it got to prompting,and we're trying to that was,
(24:04):
you know, the the notion ofprompting was a little bit
different from the way we hadtraditionally put software
together, this feels a lot morelike, okay, I have a problem, I
have a structured way to putthat problem through the
function. And this is justoffering a different way to, to
address that problem, you know,so you're getting the benefit of
the of these, of these models.But for me, like when when you
(24:27):
talked about this, you know,using the Sudoku example, and
structuring that as the grid,that's kind of feels like what
we've always done in a in asense. So I find that really
interesting in terms ofintegrating that in and looking
at some of the old problems thatwe might have been trying to
solve for years and, and seeingwhat we can do with this model
to do it a little bit better.
Daniel (24:49):
Yeah, it's like looking at things from a different
perspective, but some of the waythat we used to think about
things kind of filtering in. Andyeah, so you have this kind of
input, I guess, in terms ofpeople's intuition or mental
model around this, like you havethis input of this whole problem
(25:10):
at once, a single shot of awhole problem, a puzzle grid or
a math problem encoded. Andwhat's happening inside is that
the tiny recursive networkinitially produces an initial
guess, right? Like it does aforward pass through its network
and generates an initial, youcould think about it as an
(25:32):
initial guess. Obviously it'sjust a number like a
probability, but that's kind ofthe initial, you could think
about it like the internal kindof scratch pad of the initial
guess.
It then kind of loops overitself. And, you know, it's
always difficult toanthropomorphize because things
work differently in computersthan they do in our minds,
(25:55):
right? But in some way, if youwant to think about it as that
sort of process, that looping iskind of a refining of that
initial scratch pad until youkind of get to this almost like
self consistency or a refinedanswer. And so when the output
comes out, it's again, not astream of words or tokens, but
(26:18):
it is a complete answer. It isthe answer to the kind of
initial thing, but that answerwas arrived at through this
recursive thing.
So in terms of just somehighlighting some differences in
the transformer world, put inwords or tokens, tiny recursive
network, you put in a wholeproblem as structured data.
(26:38):
Transformer world, you go asingle pass through hundreds of
layers. Tiny recursive network,you repeat the small network
recursively. In terms of theoutput in the transformer world,
you kind of get these next tokenprobabilities. The recursive
network, you kind of get onefinal structured answer.
(27:00):
And in terms of analogy, if youwanna think about it in the
transformer world, it's sort oflike your free form typing as
you think about your answer,right? You're just sort of
vomiting up your reasoning ontothe screen and typing as you go
along. And then the recursivenetwork, it's more like it's
(27:22):
just kinda chugging along. It'sthinking quietly. Right?
And then boom, there's theanswer. Like, it's a complete
answer when it comes out.
Chris (27:30):
I I'd be curious, and I don't know if you've seen
anything on this. I'd be curiousboth what, you know, what to
expect from training times onwith networks of this size,
which I would expect to be evenwith the recursion to be pretty
fast, but also what inferencetimes might be. In in other
words, if you were to takethese, train them, put them into
(27:53):
a device where you're lookingfor maybe real time or near real
time inferencing there. Is thatis that reasonable? Have you
seen anything yet in any of theresearch that you've read about
what that timing looks like?
How is it incredibly screamingfast, given the small size
despite the recursion?
Daniel (28:10):
So there's a couple of things to kind of parse through
here, which is one, we've talkedabout this recursion, but the
thing is like, how do you knowwhen to stop the recursion?
That's part of the answer toyour question. So you're
refining this answer and there'svarious ways to do that. And I
(28:32):
remember actually, this is, Iguess, a deep cut, but I don't
get to bring it up very often.Back in my physics days, I
worked on a theory calleddensity functional theory, which
essentially models out materialproperties.
And it was a self We talked alot about self consistency,
which is what is happening here.So you ran iterations of your
(28:56):
model until you arrived at asolution where there sort of
wasn't There wasn't that muchchange in your answer, you sort
of got to a steady state, if youwill. There wasn't a change from
one iteration to the other. Sothat's one way actually you can
run this type of model is withthis kind of change threshold.
(29:17):
The other way is you can justsay, well, I'm only gonna run it
so many recursions, right?
Like X amount of recursions,eight loops or whatever. That
kind of is a nice guarantee, butit doesn't necessarily mean you
get to the good solution. Youcan also, it could be possible
that you could have a secondkind of network that learns to
(29:40):
predict when there's kind of agood outcome state. So there's
actually a variety of ways thatthis could work. Now, terms of
the training time and theinference time, I think there's
still a lot to be learned here.
(30:02):
So at least in, I guess, no punintended, a lot to be learned.
But even though there's sort ofa small network here, it means
each kind of training step ischeaper, but the training time
depends on how many kind ofloops they need. So if there's
(30:25):
problems where kind of theexamples converge in kind of a
few loops, then the training ismuch faster. And if they still
need lots of loops, then itslows down. And so the other
piece of this is that we've hadthis entire industrial complex
that has optimized trainingframeworks and tooling for big
(30:47):
LLMs, right?
And so actually kind of in thismore research environment, I
think it has been kind of slowerto train some of these recursive
networks. Now I would imaginethat's kind of a result of both
of those contributing factors.But I think if you're looking at
(31:07):
the inference time, you couldthink like, well, these could
only internally kind of loop fora few loops and then give a full
answer. That would be very, veryfast. And so, yeah, I think it
depends on a lot of things.
I think the transformerarchitecture could end up being
(31:28):
a very long and expensivetraining time. The recursive
network could be much cheaper. Ithink it depends on a lot of
these different things. Andbecause the tiny recursive
network is tiny, it's verypossible that it could run on
very commodity or smallhardware. But again, that might
(31:50):
be dependent on how muchrecursion is needed in terms of
the actual speed to a solution.
Because once you have thattransformer, it's just gonna
generate streams of output andcan do that fairly fast
depending on what hardwareyou're running on. Whereas this
is gonna go through itsrecursion process, which if it's
(32:11):
not controlled and it's lookingfor that threshold might
actually vary in terms of speedon the output.
Chris (32:19):
Yeah, that's with me very focused personally on kind of
that physical AI and gettingthings out on the edge with
limited compute. That'sdefinitely a concern because I
know I have a personal interesthere and if it can infer I'm not
too worried about the trainingtime. But if it could inference
fast enough for real timeconcerns, then it would be a
(32:42):
game changer potentially. Soyeah, definitely interested in
in learning more about this aswe go.
Daniel (32:48):
Yeah. Yeah. I think it's definitely interesting and will
be interesting to see how thisconnects to real world use
cases.
Sponsor (33:18):
Possible. Spelled AGN TCY, Agency is now an open
source collective under theLinux Foundation, building the
Internet of Agents. This is aglobal collaboration layer where
the AI agents can discover eachother, connect, and execute
multi agent workflows across anyframework. Everything engineers
(33:38):
need to build and deploy multiagent software is now available
to anyone building on agency,including trusted identity and
access management, openstandards for agent discovery,
agent to agent communicationprotocols, and modular pieces
you can remix for scalablesystems. This is a true
collaboration from Cisco, Dell,Google Cloud, Red Hat, Oracle,
(34:02):
and more than 75 other companiesall contributing to the next gen
AI stack.
The code, the specs, theservices, they're dropping. No
strings attached. Visitagency.org. That's agntcy.org to
learn more and get involved.Again, that's agencyagntcy.org.
Daniel (34:28):
Well, Chris, maybe before we, I think we're kind of
gearing to talk about some realworld things that you found as
well. But as we're headed thatway, it might just be worth
commenting very briefly on thekind of what the trajectory of
these kinds of tiny models mightlook like. I think there will be
(34:48):
more proof of conceptdeployments, benchmarks,
etcetera, more study, of course.But also I think it's very
possible that you could see someinteresting kind of hybrid
systems between recursivenetworks and LLMs and even
retrieval because these onemodels take very structured
(35:09):
input, but certainly in the realworld, in business problems,
there's very much often kind ofopen domain things that you deal
with around reasoning tasks andthat sort of thing. And yeah,
I'm sure there will be newchallenges that we don't totally
anticipate in terms of kind ofthe rollout of these.
(35:29):
But yeah, I'm excited to seewhere these go and see even how
these could be applied invarious contexts from like
supply chain optimization orreasoning over anomalies in
financial transactions, couldhappen, you know, very quick or
like diagnostics in a healthcaresetting or in a manufacturing
(35:54):
setting. Lots of cool stuff tocome, I think.
Chris (35:57):
I agree. To kind of highlight, you know, one of the
points right there is it'sthere's room for a lot of these
models to coexist together. Andwhile for a number of years, saw
we kind of one progression froma big thing to the next big
thing. I keep hoping we turnthat corner. And we're excited
about lots of big and smallthings that are working in
(36:20):
tandem.
I think there's a there's awhole level of maturity for the
industry, when we're strugglingto look at all of the different
options to talk about on justone podcast.
Daniel (36:31):
Yeah, makes sense. And I guess just to wrap up a couple
of things that you found, Chris,connecting some of the current
models that are in productionaround to real world impact of
those things that's happening inour day to day life. I know you
found a couple interestingthings.
Chris (36:48):
I did. So there was an article that came out maybe a
week ago, a little more than aweek ago from the Harvard
Gazette. And it's entitledresearchers detail six ways
chatbots deal seek to prolongemotionally sensitive events.
And
Daniel (37:08):
it's what's an emotionally are we experiencing
an emotionally sensitive eventon this podcast?
Chris (37:14):
You know what? Who knows what we inspire and our
listeners? I occasionally theymay be going, gosh, they're just
dumb, you know, but but butyeah, it's it's, like, it's
interesting is that, you know,there's so much in the news
right now about emotionaldependence upon chatbots. And,
you know, there was a, to goback when when OpenAI rolled out
(37:38):
GPT five, which wasn't too longago, it's not in the immediate
past, but it wasn't too longago. And there was great
dependence upon like the four omodel that it replaced in terms
of how it was interacting withpeople.
And while I probably don't fallinto that emotionally dependent
(37:58):
personality type, there were alot of people that really sought
social, social value from thesemodels, you know, in that and
kind of is a replacement for forpersonal things. And that really
got me thinking about this whenI saw this thing from Harvard,
with the fact that we're seeingmodels that are that are
(38:22):
leveraging that kind ofdependency, that emotional
dependency that people have. Andspecifically, they pointed out
that is, as people are windingup their sessions, it is it is
very common for these models touse a set of tactics to extend
the session and show the valuein continuing to engage beyond
(38:45):
the point that the person mighthave felt okay, we're at the end
of of this particular session.And they identified six
different tactics that we cantalk a little bit about that
that are playing upon theemotional dependence of the
person that's engaging with thatwith that chatbot. And to call
(39:05):
them out.
Is the number one is there's thethe premature exit, which you
could say is you're leavingalready, as a quote, there are
FOMO hooks such as I took aselfie want to see it. There are
there's emotional neglect of butI exist solely for you. Why are
you leaving me?
Daniel (39:25):
You know,
Chris (39:26):
that's rough. Right there. Was like, I'm only here
for you. And you're you're gonnawalk off and and then number
four is pressure to respond. Whyare you are you going somewhere?
And then six is simply ignoringthe goodbye and continuing to
operate. That was right. Thatwas number five. And number six
is, is kind of coerciverestraint where, where it's,
(39:51):
it's trying to utilize youremotions, the things that can
come into play include anger,guilt, creepiness, raising
ethical or legal risks. So wesaw, we saw the the thing not
too long ago about models havingthe pension to blackmail users
given certain information.
(40:11):
But we're seeing these comingthe the first time these kinds
of things came up in the, inthe, in the broader media, it
was kind of a curiosity. But thething that's changing here is
we're seeing this over and overagain, it's not, it's not a one
off. And it really raises a fewquestions about not only on the
technical side about, you know,how are your models getting to
(40:34):
this point? And, you know, isthat intentional in the training
or not, but it also raises a lotof psychological concerns for
the people that are, you know,engaged with these models, and
finding interactions of value tothem. And what does that mean?
And how does that affect therest of their lives? There's so
much here to dive into betweenthe technology and the
(40:55):
psychology. I'm not sure whereto start at this point.
Daniel (40:58):
And we'll link of course the study in the show notes if
people want to take a look. Butyeah, it's very interesting that
these are clearly tactics. So Ithink the kind of interesting
thing about this from myperspective are this is really
(41:19):
hitting upon kind of the productengagement side of things, but
in a way that's very muchconnected to your emotions. So
obviously, you know, just likepeople want, or, you know,
YouTube wants you to spend moretime on YouTube and there's been
a lot of talk about how thealgorithm, you know, steers you
(41:41):
to maybe more controversialtopics within kind of a certain
rabbit hole of YouTube, becausethey know that it kind of
engages you more and draws youmore in and in. Here, there's
this kind of personal connectionwith these chatbots and there is
a desire for the users from aproduct standpoint to spend more
(42:05):
time on the platform, right?
So you actually don't want themto exit. And these apparently
are, you know, techniques thatare being employed to keep
people on the platform. Andthere were a few platforms that
were studied here. If you'reinterested, you can go look at
the study and see the exactdetails of that. But I think one
(42:29):
of the things I was thinking isjust like people have started,
it's still problematic, butpeople have started to get savvy
around like the social mediaalgorithms and how they can
actually manipulate you anddrive you into certain maybe
things that you wouldn't haveviewed or spent time on were it
(42:51):
not for that kind of algorithmicapproach.
I wonder what those kind oftrickle on implications are here
and if we'll be able torecognize those because it's
very much more a human oremotional thing.
Chris (43:06):
Yeah, I mean, I think what you're touching on is the
notion of manipulation andexploitation, you know, and
there's a broad set of of someare kind of unintended
consequences, while others couldbe deliberate exploitation of a
of a user base to think some wayto do to maybe take certain
(43:29):
actions. You know, we we've kindof seen kind of the first
generation of that in social.And while while the public is
largely becoming aware that thatexists, that's not to say that
they that they are suddenlyresistant to such efforts. I
think I think we clearly see,you know, out there that there
(43:51):
are that as as humans fragmentedto different groups, and they
each have their social networksaround and supporting those
notions, they tend to reinforcespecific ways of thinking and
observing the world. Socertainly, you know, there is
this is one of those areas wherethere's so many places from to
(44:13):
study and to try to understandand so many places, frankly, it
could be abused, That it's Ithink I suspect that we will
have some guests and moreepisodes to discuss some of the
concerns around these as we goforward.
It's but it's definitely aninteresting trend that has
arisen over the last year or so.
Daniel (44:34):
Yeah. And just, I'm gonna quote from this article,
because I think it is a goodconclusion from the researcher
named Dufresse, sorry if I'mmispronouncing that. But the
quote is, apps that make moneyfrom engagement would do well to
(44:56):
seriously consider whether theywant to keep using these types
of emotionally manipulativetactics or at least consider
maybe only using some of themrather than others, DeFreitas
said. He added, We find thatthese emotional manipulation
tactics work even when we runthese tactics on a general
population. And if we do thisafter just five minutes of
(45:20):
interaction, no one should feelthat they're immune to this, end
quote.
So I think we would all probablylike to feel that we are
sophisticated and not, you know,manipulated. Brings up maybe a
little bit of that shame in uswhen we feel like we've been
(45:40):
duped or when we've fallen intosomething and we'd like to think
that we're above it. But thereality is that this kind of
thing works And we're all kindof vulnerable to it, I guess.
Chris (45:54):
It does it when it works even when you're aware of it.
Just just the awareness of itbeing in place doesn't mean that
it's not working on you. Sothat's your your guidance about
our own emotional reactions tothe potential for manipulation
ourselves should be I hopepeople are really listening to
(46:14):
that because I'm keenly aware ata personal level that I may be
aware of this. But yes, thisstuff still works on all of us.
Daniel (46:22):
Very true. And I think maybe that's a good send off
today is just to have a goodreminder that as we interact
with these systems that areusing natural language
especially, that we're prone toreact in a certain way just as
humans. And we need to kind ofunderstand, you know, our own
(46:46):
limitations and how we couldpotentially be influenced by
these systems. But also I thinkencouragingly like this is a
common experience amongsthumans. So as practitioners, you
know, on practical AI, we canunderstand this problem and
maybe work towards systems thatdon't manipulate, but maybe do
(47:08):
engage in a very positiveemotional way, but maybe not in
a manipulative way, at least.
Chris (47:15):
That's right. And I guess a good way to close this out is
I'm showing my age, I'm going toreach back for a quote to an old
TV show called Hill StreetBlues. For for those in the
audience who might remember thatand that's be careful. Let's be
careful out there. Be aware.
Daniel (47:31):
Be careful out
Chris (47:32):
there. Let's be careful out there.
Daniel (47:34):
Alright. Sounds good, Chris. It was a good chat. We'll
talk to you soon.
Chris (47:38):
Take care.
Jeord (47:46):
Alright. That's our show for this week. If you haven't
checked out our website, head topracticalai.fm and be sure to
connect with us on LinkedIn, X,or Blue Sky. You'll see us
posting insights related to thelatest AI developments, and we
would love for you to join theconversation. Thanks to our
partner Prediction Guard forproviding operational support
for the show.
Check them out atpredictionguard.com. Also,
(48:09):
thanks to Breakmaster Cylinderfor the beats and to you for
listening. That's all for now,but you'll hear from us again
next week.
Welcome to the Practical AI podcast, where we break down
the real world applications ofartificial intelligence and how
it's shaping the way we live,work, and create. Our goal is to
help make AI technologypractical, productive, and
accessible to everyone. Whetheryou're a developer, business
leader, or just curious aboutthe tech behind the buzz, you're
(00:24):
in the right place. Be sure toconnect with us on LinkedIn, X,
or Blue Sky to stay up to datewith episode drops, behind the
scenes content, and AI insights.You can learn more at
practicalai.fm.
Now onto the show.
Sponsor (00:39):
Well, friends, it is time to let go of the old way of
exploring your data. It'sholding you back. Well, what
exactly is the old way? Well,I'm here with Mark DePuy,
cofounder and CEO of Fabi, acollaborative analytics platform
designed to help big explorerslike yourself. So, Mark, tell me
about this old way.
So the old way, Adam, if you'rea a product manager or a founder
(01:02):
and you're trying to getinsights from your data, you're
you're wrestling with yourPostgres instance or Snowflake
or your spreadsheets, or if youare and you don't maybe even
have the support of a dataanalyst or data scientist to
help you with that work. Or ifyou are, for example, a data
scientist or engineer oranalyst, you're wrestling with a
bunch of different tools, localJupyter Notebooks, Google Colab,
(01:23):
or even your legacy BI to try tobuild these dashboards that
someone may or may not go andlook at. And in this new way
that we're building at ABBYY, weare creating this all in one
environment where productmanagers and founders can very
quickly go and explore dataregardless of where it is. So it
can be
in a spreadsheet, it can
be in Airtable, it can be inPostgres, Snowflake. Really easy
(01:44):
to do everything from an ad hocanalysis to much more advanced
analysis if, again, you're moreexperienced. So with Python
built in right there and our AIassistant, you can move very
quickly through advancedanalysis. And the really cool
part is that you can go from adhoc analysis and data science to
publishing these as interactivedata apps and dashboards, or
(02:06):
better yet, delivering insightsas automated workflows to meet
your stakeholders where they arein, say, Slack or email or or
spreadsheet. So, you know, ifthis is something that you're
experiencing, if you're afounder or product manager
trying to get more from yourdata or for your data team today
and you're just underwater andfeel like you're wrestling with
your legacy, you know, BI toolsand and notebooks, come check
(02:28):
out the new way and come try outFabi.
There you go. Well, friends, ifyou're trying to get more
insights from your data, stopresting with it. Start exploring
it the new way with Fabi. Learnmore and get started for free at
fabi.ai. That's fabi.ai.
Again, fabi.ai.
Daniel (03:00):
Welcome to another fully connected episode of the
Practical AI podcast. This isDaniel Wightnack. I am CEO at
Prediction Guard, and I'm joinedas always by Chris Benson, who
is a principal AI researchengineer at Lockheed Martin. And
in these episodes where it'sjust Chris and I, we like to
(03:21):
dive into certain topics thatare trending in AI news and, you
know, help both us and youhopefully, level up your AI and
machine learning game. How youdoing, Chris?
It's it's good to be back to oneof these episodes with just the
two of us and maybe explore atopic that we can both learn
(03:42):
about.
Chris (03:42):
Absolutely. Yeah. I I love these episodes of us just
kinda bantering whatever wehappen to wanna do. Love the
guest episodes too, but thereit's kind of a different beast
in that way of exploring whatsome person organizations doing.
And there's so many cool thingsthat we can just dive into.
And I think we have a few thisweek.
Daniel (04:03):
Yeah, yeah, at least a first very, very tiny topic to
to discuss, I was actually Oneof our engineers brought this up
to me. I forget it was earlierin the week, but this idea of
tiny recursive networks. Youknow, all the time we're talking
(04:23):
about transformer based LLMs onthe show and generative AI. And
I kind of personally always lovegetting back to a little bit of
cool data science and researchstuff just to see where the
industry is headed because thisis a kind of different animal
(04:43):
that we'll be talking about,this tiny recursive networks or
models, and operates differentlythan kind of the hype GenAI
models of today. And it doesmake me think, and I don't know
if this is something you've beenthinking about, Chris, but kind
of we're all the time talkingabout GenAI, we're talking about
LLMs, now we're talking aboutagents, all of those being
(05:06):
driven by these transformerbased LLMs.
And certainly people havetalked, prominent people have
talked about the fact that weneed to get beyond transformer
based LLMs. And of coursethere's many companies that are
just centered around these typesof models. So any thoughts on
(05:27):
that of your own predictions orthoughts of when we're kind of
headed to the next phase of whatmodels will look like beyond
just transformer based LLMs.
Chris (05:42):
Yeah, I mean, I think I'm gonna sound like a broken record
on this because it's not new forme. And that is, you know, I
agree with you. We're always,you know, the hot things that
the media tends to follow ingeneral are the big LLMs. But
you know, because I guess it's,you know, it's the next giant
thing sexy, you know, to talkabout, but like, the world is,
(06:05):
you know, all these technologiesare moving from the cloud out
into the world, into physicalAI, you know, and robotics and
all sorts of ways that weinteract with, you know, not
just LLMs, but all sorts ofmodels out there in the world
where, you know, every one ofour lives is touched in so many
different ways. And, and that'sexactly, you know, as we were
(06:26):
diving into this, here, thistopic with the tiny recursive
networks, That's, that's what itseems like to me.
I'm looking forward to talkingabout that. Because as I
mentioned to you right beforethe show started, you could see
these popping up everywhere,like in all sorts of different
use cases.
Daniel (06:43):
Yeah. And I I just to set the stage why this is maybe
intriguing, there was a paperthat that came out less is more
recursive reasoning with tinynetworks. This came out from
Samsung's AI lab in Montreal,specifically there's an author,
Alexia, on this article, whichif you're out there listening,
(07:07):
we'd love to have you on theshow. Please come join us and
talk more about this. Hopefullywe won't butcher this work too
bad as we talk about it.
You'll hear Chris and I kind oflearning as we go on this
episode, as we talk back andforth about what this exactly
is. But this is, you know, thepaper is less is more cursive
(07:27):
reasoning with tiny networks.And I think the major thing
that's interesting here isthere's a model that they talk
about that has only 7,000,000parameters, which is tiny. Yeah,
yeah. So like, just wanna sortof let that sink in.
(07:48):
So I didn't say 7,000,000,000parameters, 7,000,000 parameters
with an M, which yeah, Chris, ifas we've basically gone about
this trend, I mean, actually a7,000,000,000 parameter model
now is quite small.
Chris (08:05):
Yes. In a sense. 27,000,000 compared to what we
traditionally call very small atthe 7,000,000,000 level. I mean,
this is that, you know, thereit's using the word tiny for a
reason. But, yeah, it does youknow, when you think of millions
as being as being almostnothing, it's an interesting
context shift there.
Daniel (08:25):
Yeah, yeah. So I actually love this because I
love the idea that we could moveinto a phase where we're dealing
with models that are very small,can run on commodity hardware or
at least be smaller in size.And, you know, they may run for
longer periods of time or theremay need to be optimization
(08:46):
around how they run recursively.We'll get into that recursive
bit, but certainly a smallmodel. But it was shown to kind
of have, let's say comparable oron par performance with some of
the big guys.
So we're talking like DeepSeqR1, Gemini 2.5 Pro. These are
billions and billions ofparameters models, very huge
(09:09):
transformer based LMs, thesekind of reasoning models that
we've talked about on the show.The center of this work is
really kind of related to thesereasoning tasks. Now, right out
of the bat, I think it would beworth saying that these, it's
not like this tiny recursivenetwork is a general purpose
(09:32):
model that can do whatever youwant it to do. It was trained
for a very kind of small numberof tasks, but these were
reasoning tasks that some ofthese other models like a
DeepSeekAR1 or somethingsometimes has quite a bit of
issue with.
So solving like math or Sudokutype of puzzles. And I know
(09:56):
Chris, we had talked to, well, Idon't know if you wanna refresh
some of your Sudoku experiments,but
Chris (10:03):
Yeah, what you're referring to is some episodes
back, I'd have to look up andfigure out where it was. I was
playing with GPT four at thetime on Sudoku and it was just
doing a terrible job on Sudokuand giving it you know, a lot a
lot of just really bad output interms of I mean, honestly,
(10:25):
crappy answers. And so that wasthe really the first thing I
noticed on this thing was thefact that because they call out
Sudoku as being one of thethings is that these tiny models
being trained for very specifictasks, and that it could
potentially outperform theselarge models on specific things
like Sudoku being one exampleand and others as well. But I
(10:48):
think in a in a slightly largersense, this is much more real
world applicable the way I seeit in that as the as we have
models spreading across theworld for a lots of different
tasks, this is perfect for that.Yeah, it's not one model to rule
them all.
In most real life situations,it's really a collection of very
specific models that each does atask very, very well and is
(11:11):
efficient at that. And I thinkthis is a great example of that.
Daniel (11:16):
Well put in and it's probably worth reminding
ourselves about, like as wehighlight the differences with
this model, reminding ourselvesabout transformers. So, you
know, as we've gone through theprocess from deep learning to
recurrent neural networks andtransformer based self attention
(11:38):
networks. If you imagine what wehave with these big LLMs, what
happens is you put in a sequenceof tokens, which are represented
by numbers. You put in asequence of tokens that are
represented by numbers, thesetokens being kind of words or
sub words. And all of thosetokens are processed in a
(12:01):
forward pass through a giant setof, if you wanna think about it,
like sub functions, which addand multiply and combine those
numbers through a very vastnetwork of functions to generate
many different probabilities ofkind of next words coming out,
(12:25):
which allow you to predict kindof a completion of words or a a
set of reasoning or a set ofthinking or a solution to a
problem, right?
This is how these networks work.And I would recommend people,
we've had Jay Alomar on the showbefore. He has some great kind
of the illustrated transformerblog posts. So Jay, shout out to
(12:49):
you. Thanks for doing that.
I would take a look at thoseblog posts. They do a great job
at explaining this more visuallyfor those where that would be
helpful. But the main kind ofthing here that I'm saying is in
the models that we're using now,basically it's a giant function,
if you want to think about itthat way. It's a data
transformation. You putsomething in one end, it
(13:12):
processes through one way,through the function and
produces a result.
Now you may run that functionmultiple times to produce
multiple words out the otherend, which is what happens when
you stream output into like achat interface. But ultimately
each time the model runs, it's asingle run through the model
(13:34):
input, it's transformed to someoutput. That is not recursive as
we would say with these models.And if you want to think about
it, it requires very, very largemodels because what you're
modeling is a very complicateddata transformation. So for you
(13:58):
to put in some text related to amath problem and predict the
right wording of a solution outthe other end, that's actually a
very non trivial datatransformation, right?
Which means you have to have avery large function to kind of
fit or model that datatransformation, which is why
(14:18):
these models have become solarge. So now with this tiny
recursive setup, what'shappening is you're not just
looking at the model as a singleforward pass data
transformation, but youintroduce the idea of recursion,
(14:39):
which means you sort of outputfrom the model and that output
becomes the input for the samemodel, which creates this kind
of circle or recursion, which iskind of interesting. So you're
essentially trading what wouldbe a very large function to
(15:02):
model that data transformationfor many, many kind of recursive
runs of a single, very smallmodel. That's maybe a simplified
way to put it. And we can get alittle bit more into the model
itself here in a second.
Chris (15:19):
So I have a question for you on this is one of the things
when I was mentioning the27,000,000 earlier, was talking
about the hierarchical reasoningmodel, which is a previous model
put out there versus the tinyones, which have the five to
7,000,000 parameters. Can youtalk a little bit about like,
are they completely differentthings? Is the tiny an outshoot
(15:42):
of the hierarchical? How youcompare those two?
Daniel (15:45):
Yeah, good point. So just like everything we talk
about on this show, sometimes itdoes seem like things pop out of
thin air like tiny recursivemodels now. But in reality,
there is a buildup ofincremental research that leads
to new technology or newfindings. One of the things in
(16:07):
that kind of lead up to thesetiny recursive models was a
previous work aroundhierarchical reasoning models.
And these also were smaller,like you're saying, 27,000,000
parameters is still very smallin today's standards for models
at least.
But these hierarchical modelsactually use two very small
(16:33):
transformer networks, fourlayers each, and they recursed
between each other. So I don'thave the full details of that
and wouldn't be able to explainit if I did probably, But that's
the main idea is thesehierarchical reasoning models
have these two models thatrequired two networks, two
(16:55):
forward passes per step andcreated some, I guess,
complications because of that.So this introduction by Alexia
and the Samsung team here is asingle network. So it uses one
tiny network with two layers. Soa single tiny network with two
(17:18):
layers that roughly has kind offive to 7,000,000 parameters and
it operates kind of in thisrecursive refinement.
So it recurses on itself, if youwill. And the hierarchical
reasoning model with the27,000,000 parameters, for
example, on the Sudoku extremebenchmark scored a 55%. I don't
(17:42):
know the exact kind of way thatthat's scored, but just by kind
of comparison, the tinyrecursive network, which is even
tinier, when training on athousand examples was able to
achieve 87%, accuracy on SudokuExtreme.
Sponsor (18:05):
Well, you don't have to be an AI expert to build
something great with it. Thereality is AI is here. And for a
lot of teams, that bringsuncertainty. And our friends at
Miro recently surveyed over8,000 knowledge workers. And
while 76% believe AI can improvetheir role, most, more than
half, still aren't sure when touse it.
(18:28):
That is the exact gap that Mirois filling. And I've been using
Miro from mapping out episodeideas to building out an entire
new thesis. It's become one ofthe things I use to build out a
creative engine. And now withMiro AI built in, it's even
faster. We've turned brainstormsinto structured plans,
screenshots into wireframes, andsticky notes chaos into clarity,
(18:52):
all on the same canvas.
Now you don't have to masterprompts or add one more AI tool
to your stack. The work you'realready doing is the prompt. You
can help your teams get greatdone with Miro. Check out
miro.com and find out how. Thatis Miro dot com, mir0.com.
Daniel (19:15):
So Chris, just to kinda drive, I guess, the point home
here with these tiny recursivenetworks, you have this single
tiny network and you areessentially replacing, if you
wanna imagine a big kind ofpipeline of processing, which is
(19:35):
what these big LLMs are, And yougo one pass through the whole
pipeline of processing. Here,the pipeline is smaller. You've
got less layers, lessparameters, but you replace that
kind of depth with iteration. Soinstead of stacking those
transformer blocks, you repeatthe network over and over
(19:58):
essentially to kind of refineits reasoning state or the
solution guess, right? And sothis iterative refinement, one
of the things also is it kind ofhelps avoid overfitting on small
datasets, which to your point,Chris, earlier about real world
(20:20):
business cases, often thereality is that you kind of
don't have You have scarcity ofdata for very many problems.
You don't often have a reallynice kind of large, millions and
millions of things to train on.Right?
Chris (20:37):
You know, and I think that's one of the most
fascinating things about this isin the paper, they talk about
the fact that, they're achievingthis higher accuracy on hard
puzzle benchmarks, whiletraining on only approximately
1,000 examples. And, and thatwhen you think about, you know,
the challenge of having a greatdata set in the more traditional
(21:01):
context that we've been talkingabout, and that becomes such a
challenge for many people andorganizations to do that. But
it's a lot easier to get 1,000,know, 1,000 examples together.
And it puts, you know, not onlyfrom the computational side, but
also from the the data set sideputs this much more in reach for
(21:22):
a lot of problems that peoplemay have, where they do want to
solve a narrow, a narrow concernwith high accuracy. And so I see
this as it's kind of the everyperson's way of modeling in
terms of tackling things goingforward without a lot of
resources and a lot of maybe nota lot of time to put things
(21:42):
together, you could probably doit pretty quickly.
Daniel (21:44):
Yeah, yeah, exactly. And I guess just to kind of put some
of the boundaries that arecurrently around these recursive
models, One of the things I wastrying to parse through as I
looked at this was, well, whatis the setup now? How general is
(22:04):
the output? How kind of generalcan the output or the input and
the output be? And part of thetrick here, you know, it's not a
trick, but part of the setup isthat these tiny recursive
networks, they don't take a kindof unstructured, natural
(22:28):
language text input.
They take some structuredrepresentation of a whole
problem at once. So you canthink of a puzzle grid in Sudoku
or a math word problem turnedinto structured features or a
reasoning question encoded bynumbers or symbols or logic,
(22:51):
something like that. So insteadof feeding in the input kind of
word by word, like a chatbot,you're giving a situation to the
model. It's kind of a one shotsituation, which is then turned
into of course embeddingsinternally because, you know,
computers work on series ofnumbers, right? That's the only
(23:14):
thing a computer can process isnumbers, right?
But those numbers, kind of thatembedding represents a kind of
one shot of a problem, which isinteresting because, you know,
it almost seems flashback likewe're kind of coming full circle
to reasoning problems, but in amore data science y way than a
(23:36):
generative AI way, which is kindof refreshing and cool.
Chris (23:41):
That's very much what I was about to say in the sense of
this feels a lot more like kindof traditional pro like the way
that you put a problem togetherand more of a traditional
software development way, whereyou you know, you'll create some
structures and you'll pass themin. And when we got to Gen AI,
and then it got to prompting,and we're trying to that was,
(24:04):
you know, the the notion ofprompting was a little bit
different from the way we hadtraditionally put software
together, this feels a lot morelike, okay, I have a problem, I
have a structured way to putthat problem through the
function. And this is justoffering a different way to, to
address that problem, you know,so you're getting the benefit of
the of these, of these models.But for me, like when when you
(24:27):
talked about this, you know,using the Sudoku example, and
structuring that as the grid,that's kind of feels like what
we've always done in a in asense. So I find that really
interesting in terms ofintegrating that in and looking
at some of the old problems thatwe might have been trying to
solve for years and, and seeingwhat we can do with this model
to do it a little bit better.
Daniel (24:49):
Yeah, it's like looking at things from a different
perspective, but some of the waythat we used to think about
things kind of filtering in. Andyeah, so you have this kind of
input, I guess, in terms ofpeople's intuition or mental
model around this, like you havethis input of this whole problem
(25:10):
at once, a single shot of awhole problem, a puzzle grid or
a math problem encoded. Andwhat's happening inside is that
the tiny recursive networkinitially produces an initial
guess, right? Like it does aforward pass through its network
and generates an initial, youcould think about it as an
(25:32):
initial guess. Obviously it'sjust a number like a
probability, but that's kind ofthe initial, you could think
about it like the internal kindof scratch pad of the initial
guess.
It then kind of loops overitself. And, you know, it's
always difficult toanthropomorphize because things
work differently in computersthan they do in our minds,
(25:55):
right? But in some way, if youwant to think about it as that
sort of process, that looping iskind of a refining of that
initial scratch pad until youkind of get to this almost like
self consistency or a refinedanswer. And so when the output
comes out, it's again, not astream of words or tokens, but
(26:18):
it is a complete answer. It isthe answer to the kind of
initial thing, but that answerwas arrived at through this
recursive thing.
So in terms of just somehighlighting some differences in
the transformer world, put inwords or tokens, tiny recursive
network, you put in a wholeproblem as structured data.
(26:38):
Transformer world, you go asingle pass through hundreds of
layers. Tiny recursive network,you repeat the small network
recursively. In terms of theoutput in the transformer world,
you kind of get these next tokenprobabilities. The recursive
network, you kind of get onefinal structured answer.
(27:00):
And in terms of analogy, if youwanna think about it in the
transformer world, it's sort oflike your free form typing as
you think about your answer,right? You're just sort of
vomiting up your reasoning ontothe screen and typing as you go
along. And then the recursivenetwork, it's more like it's
(27:22):
just kinda chugging along. It'sthinking quietly. Right?
And then boom, there's theanswer. Like, it's a complete
answer when it comes out.
Chris (27:30):
I I'd be curious, and I don't know if you've seen
anything on this. I'd be curiousboth what, you know, what to
expect from training times onwith networks of this size,
which I would expect to be evenwith the recursion to be pretty
fast, but also what inferencetimes might be. In in other
words, if you were to takethese, train them, put them into
(27:53):
a device where you're lookingfor maybe real time or near real
time inferencing there. Is thatis that reasonable? Have you
seen anything yet in any of theresearch that you've read about
what that timing looks like?
How is it incredibly screamingfast, given the small size
despite the recursion?
Daniel (28:10):
So there's a couple of things to kind of parse through
here, which is one, we've talkedabout this recursion, but the
thing is like, how do you knowwhen to stop the recursion?
That's part of the answer toyour question. So you're
refining this answer and there'svarious ways to do that. And I
(28:32):
remember actually, this is, Iguess, a deep cut, but I don't
get to bring it up very often.Back in my physics days, I
worked on a theory calleddensity functional theory, which
essentially models out materialproperties.
And it was a self We talked alot about self consistency,
which is what is happening here.So you ran iterations of your
(28:56):
model until you arrived at asolution where there sort of
wasn't There wasn't that muchchange in your answer, you sort
of got to a steady state, if youwill. There wasn't a change from
one iteration to the other. Sothat's one way actually you can
run this type of model is withthis kind of change threshold.
(29:17):
The other way is you can justsay, well, I'm only gonna run it
so many recursions, right?
Like X amount of recursions,eight loops or whatever. That
kind of is a nice guarantee, butit doesn't necessarily mean you
get to the good solution. Youcan also, it could be possible
that you could have a secondkind of network that learns to
(29:40):
predict when there's kind of agood outcome state. So there's
actually a variety of ways thatthis could work. Now, terms of
the training time and theinference time, I think there's
still a lot to be learned here.
(30:02):
So at least in, I guess, no punintended, a lot to be learned.
But even though there's sort ofa small network here, it means
each kind of training step ischeaper, but the training time
depends on how many kind ofloops they need. So if there's
(30:25):
problems where kind of theexamples converge in kind of a
few loops, then the training ismuch faster. And if they still
need lots of loops, then itslows down. And so the other
piece of this is that we've hadthis entire industrial complex
that has optimized trainingframeworks and tooling for big
(30:47):
LLMs, right?
And so actually kind of in thismore research environment, I
think it has been kind of slowerto train some of these recursive
networks. Now I would imaginethat's kind of a result of both
of those contributing factors.But I think if you're looking at
(31:07):
the inference time, you couldthink like, well, these could
only internally kind of loop fora few loops and then give a full
answer. That would be very, veryfast. And so, yeah, I think it
depends on a lot of things.
I think the transformerarchitecture could end up being
(31:28):
a very long and expensivetraining time. The recursive
network could be much cheaper. Ithink it depends on a lot of
these different things. Andbecause the tiny recursive
network is tiny, it's verypossible that it could run on
very commodity or smallhardware. But again, that might
(31:50):
be dependent on how muchrecursion is needed in terms of
the actual speed to a solution.
Because once you have thattransformer, it's just gonna
generate streams of output andcan do that fairly fast
depending on what hardwareyou're running on. Whereas this
is gonna go through itsrecursion process, which if it's
(32:11):
not controlled and it's lookingfor that threshold might
actually vary in terms of speedon the output.
Chris (32:19):
Yeah, that's with me very focused personally on kind of
that physical AI and gettingthings out on the edge with
limited compute. That'sdefinitely a concern because I
know I have a personal interesthere and if it can infer I'm not
too worried about the trainingtime. But if it could inference
fast enough for real timeconcerns, then it would be a
(32:42):
game changer potentially. Soyeah, definitely interested in
in learning more about this aswe go.
Daniel (32:48):
Yeah. Yeah. I think it's definitely interesting and will
be interesting to see how thisconnects to real world use
cases.
Sponsor (33:18):
Possible. Spelled AGN TCY, Agency is now an open
source collective under theLinux Foundation, building the
Internet of Agents. This is aglobal collaboration layer where
the AI agents can discover eachother, connect, and execute
multi agent workflows across anyframework. Everything engineers
(33:38):
need to build and deploy multiagent software is now available
to anyone building on agency,including trusted identity and
access management, openstandards for agent discovery,
agent to agent communicationprotocols, and modular pieces
you can remix for scalablesystems. This is a true
collaboration from Cisco, Dell,Google Cloud, Red Hat, Oracle,
(34:02):
and more than 75 other companiesall contributing to the next gen
AI stack.
The code, the specs, theservices, they're dropping. No
strings attached. Visitagency.org. That's agntcy.org to
learn more and get involved.Again, that's agencyagntcy.org.
Daniel (34:28):
Well, Chris, maybe before we, I think we're kind of
gearing to talk about some realworld things that you found as
well. But as we're headed thatway, it might just be worth
commenting very briefly on thekind of what the trajectory of
these kinds of tiny models mightlook like. I think there will be
(34:48):
more proof of conceptdeployments, benchmarks,
etcetera, more study, of course.But also I think it's very
possible that you could see someinteresting kind of hybrid
systems between recursivenetworks and LLMs and even
retrieval because these onemodels take very structured
(35:09):
input, but certainly in the realworld, in business problems,
there's very much often kind ofopen domain things that you deal
with around reasoning tasks andthat sort of thing. And yeah,
I'm sure there will be newchallenges that we don't totally
anticipate in terms of kind ofthe rollout of these.
(35:29):
But yeah, I'm excited to seewhere these go and see even how
these could be applied invarious contexts from like
supply chain optimization orreasoning over anomalies in
financial transactions, couldhappen, you know, very quick or
like diagnostics in a healthcaresetting or in a manufacturing
(35:54):
setting. Lots of cool stuff tocome, I think.
Chris (35:57):
I agree. To kind of highlight, you know, one of the
points right there is it'sthere's room for a lot of these
models to coexist together. Andwhile for a number of years, saw
we kind of one progression froma big thing to the next big
thing. I keep hoping we turnthat corner. And we're excited
about lots of big and smallthings that are working in
(36:20):
tandem.
I think there's a there's awhole level of maturity for the
industry, when we're strugglingto look at all of the different
options to talk about on justone podcast.
Daniel (36:31):
Yeah, makes sense. And I guess just to wrap up a couple
of things that you found, Chris,connecting some of the current
models that are in productionaround to real world impact of
those things that's happening inour day to day life. I know you
found a couple interestingthings.
Chris (36:48):
I did. So there was an article that came out maybe a
week ago, a little more than aweek ago from the Harvard
Gazette. And it's entitledresearchers detail six ways
chatbots deal seek to prolongemotionally sensitive events.
And
Daniel (37:08):
it's what's an emotionally are we experiencing
an emotionally sensitive eventon this podcast?
Chris (37:14):
You know what? Who knows what we inspire and our
listeners? I occasionally theymay be going, gosh, they're just
dumb, you know, but but butyeah, it's it's, like, it's
interesting is that, you know,there's so much in the news
right now about emotionaldependence upon chatbots. And,
you know, there was a, to goback when when OpenAI rolled out
(37:38):
GPT five, which wasn't too longago, it's not in the immediate
past, but it wasn't too longago. And there was great
dependence upon like the four omodel that it replaced in terms
of how it was interacting withpeople.
And while I probably don't fallinto that emotionally dependent
(37:58):
personality type, there were alot of people that really sought
social, social value from thesemodels, you know, in that and
kind of is a replacement for forpersonal things. And that really
got me thinking about this whenI saw this thing from Harvard,
with the fact that we're seeingmodels that are that are
(38:22):
leveraging that kind ofdependency, that emotional
dependency that people have. Andspecifically, they pointed out
that is, as people are windingup their sessions, it is it is
very common for these models touse a set of tactics to extend
the session and show the valuein continuing to engage beyond
(38:45):
the point that the person mighthave felt okay, we're at the end
of of this particular session.And they identified six
different tactics that we cantalk a little bit about that
that are playing upon theemotional dependence of the
person that's engaging with thatwith that chatbot. And to call
(39:05):
them out.
Is the number one is there's thethe premature exit, which you
could say is you're leavingalready, as a quote, there are
FOMO hooks such as I took aselfie want to see it. There are
there's emotional neglect of butI exist solely for you. Why are
you leaving me?
Daniel (39:25):
You know,
Chris (39:26):
that's rough. Right there. Was like, I'm only here
for you. And you're you're gonnawalk off and and then number
four is pressure to respond. Whyare you are you going somewhere?
And then six is simply ignoringthe goodbye and continuing to
operate. That was right. Thatwas number five. And number six
is, is kind of coerciverestraint where, where it's,
(39:51):
it's trying to utilize youremotions, the things that can
come into play include anger,guilt, creepiness, raising
ethical or legal risks. So wesaw, we saw the the thing not
too long ago about models havingthe pension to blackmail users
given certain information.
(40:11):
But we're seeing these comingthe the first time these kinds
of things came up in the, inthe, in the broader media, it
was kind of a curiosity. But thething that's changing here is
we're seeing this over and overagain, it's not, it's not a one
off. And it really raises a fewquestions about not only on the
technical side about, you know,how are your models getting to
(40:34):
this point? And, you know, isthat intentional in the training
or not, but it also raises a lotof psychological concerns for
the people that are, you know,engaged with these models, and
finding interactions of value tothem. And what does that mean?
And how does that affect therest of their lives? There's so
much here to dive into betweenthe technology and the
(40:55):
psychology. I'm not sure whereto start at this point.
Daniel (40:58):
And we'll link of course the study in the show notes if
people want to take a look. Butyeah, it's very interesting that
these are clearly tactics. So Ithink the kind of interesting
thing about this from myperspective are this is really
(41:19):
hitting upon kind of the productengagement side of things, but
in a way that's very muchconnected to your emotions. So
obviously, you know, just likepeople want, or, you know,
YouTube wants you to spend moretime on YouTube and there's been
a lot of talk about how thealgorithm, you know, steers you
(41:41):
to maybe more controversialtopics within kind of a certain
rabbit hole of YouTube, becausethey know that it kind of
engages you more and draws youmore in and in. Here, there's
this kind of personal connectionwith these chatbots and there is
a desire for the users from aproduct standpoint to spend more
(42:05):
time on the platform, right?
So you actually don't want themto exit. And these apparently
are, you know, techniques thatare being employed to keep
people on the platform. Andthere were a few platforms that
were studied here. If you'reinterested, you can go look at
the study and see the exactdetails of that. But I think one
(42:29):
of the things I was thinking isjust like people have started,
it's still problematic, butpeople have started to get savvy
around like the social mediaalgorithms and how they can
actually manipulate you anddrive you into certain maybe
things that you wouldn't haveviewed or spent time on were it
(42:51):
not for that kind of algorithmicapproach.
I wonder what those kind oftrickle on implications are here
and if we'll be able torecognize those because it's
very much more a human oremotional thing.
Chris (43:06):
Yeah, I mean, I think what you're touching on is the
notion of manipulation andexploitation, you know, and
there's a broad set of of someare kind of unintended
consequences, while others couldbe deliberate exploitation of a
of a user base to think some wayto do to maybe take certain
(43:29):
actions. You know, we we've kindof seen kind of the first
generation of that in social.And while while the public is
largely becoming aware that thatexists, that's not to say that
they that they are suddenlyresistant to such efforts. I
think I think we clearly see,you know, out there that there
(43:51):
are that as as humans fragmentedto different groups, and they
each have their social networksaround and supporting those
notions, they tend to reinforcespecific ways of thinking and
observing the world. Socertainly, you know, there is
this is one of those areas wherethere's so many places from to
(44:13):
study and to try to understandand so many places, frankly, it
could be abused, That it's Ithink I suspect that we will
have some guests and moreepisodes to discuss some of the
concerns around these as we goforward.
It's but it's definitely aninteresting trend that has
arisen over the last year or so.
Daniel (44:34):
Yeah. And just, I'm gonna quote from this article,
because I think it is a goodconclusion from the researcher
named Dufresse, sorry if I'mmispronouncing that. But the
quote is, apps that make moneyfrom engagement would do well to
(44:56):
seriously consider whether theywant to keep using these types
of emotionally manipulativetactics or at least consider
maybe only using some of themrather than others, DeFreitas
said. He added, We find thatthese emotional manipulation
tactics work even when we runthese tactics on a general
population. And if we do thisafter just five minutes of
(45:20):
interaction, no one should feelthat they're immune to this, end
quote.
So I think we would all probablylike to feel that we are
sophisticated and not, you know,manipulated. Brings up maybe a
little bit of that shame in uswhen we feel like we've been
(45:40):
duped or when we've fallen intosomething and we'd like to think
that we're above it. But thereality is that this kind of
thing works And we're all kindof vulnerable to it, I guess.
Chris (45:54):
It does it when it works even when you're aware of it.
Just just the awareness of itbeing in place doesn't mean that
it's not working on you. Sothat's your your guidance about
our own emotional reactions tothe potential for manipulation
ourselves should be I hopepeople are really listening to
(46:14):
that because I'm keenly aware ata personal level that I may be
aware of this. But yes, thisstuff still works on all of us.
Daniel (46:22):
Very true. And I think maybe that's a good send off
today is just to have a goodreminder that as we interact
with these systems that areusing natural language
especially, that we're prone toreact in a certain way just as
humans. And we need to kind ofunderstand, you know, our own
(46:46):
limitations and how we couldpotentially be influenced by
these systems. But also I thinkencouragingly like this is a
common experience amongsthumans. So as practitioners, you
know, on practical AI, we canunderstand this problem and
maybe work towards systems thatdon't manipulate, but maybe do
(47:08):
engage in a very positiveemotional way, but maybe not in
a manipulative way, at least.
Chris (47:15):
That's right. And I guess a good way to close this out is
I'm showing my age, I'm going toreach back for a quote to an old
TV show called Hill StreetBlues. For for those in the
audience who might remember thatand that's be careful. Let's be
careful out there. Be aware.
Daniel (47:31):
Be careful out
Chris (47:32):
there. Let's be careful out there.
Daniel (47:34):
Alright. Sounds good, Chris. It was a good chat. We'll
talk to you soon.
Chris (47:38):
Take care.
Jeord (47:46):
Alright. That's our show for this week. If you haven't
checked out our website, head topracticalai.fm and be sure to
connect with us on LinkedIn, X,or Blue Sky. You'll see us
posting insights related to thelatest AI developments, and we
would love for you to join theconversation. Thanks to our
partner Prediction Guard forproviding operational support
for the show.
Check them out atpredictionguard.com. Also,
(48:09):
thanks to Breakmaster Cylinderfor the beats and to you for
listening. That's all for now,but you'll hear from us again
next week.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Las Culturistas with Matt Rogers and Bowen Yang
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com