October 9, 2025 • 45 mins
We sit down with Dr. Subu Iyer of UCLA to unpack chiplets vs dielets, why a universal ecosystem is missing, and how sub‑10 µm bump pitch could make protocols optional. Then we host a panel featuring John Knickerbocker, IBM; Mike Kelly, Amkor; and Tolga Tekin, Fraunhofer IZM on co‑packaged optics, bandwidth, and power for AI data centers.
• chiplet as design construct, dielet as physical die
• lack of universal chiplet ecosystem and interoperability
• bump pitch scaling and protocol overhead trade‑offs
• packaging purpose reframed as power, communication, and cooling
• economic shift and value capture in advanced packaging
• national competitiveness, prototyping access, and talent pipeline
• co‑packaged optics definition, drivers, and cost targets
• copper reach limits, latency, and bandwidth density for AI
• hyperscalers as early adopters and five‑year outlook
Learn more at imaps.org
IMAPS is the largest society dedicated to microelectronics and electronics packaging advancement.
Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.
Become a sustaining member!
Like what you hear? Follow us on LinkedIn and Twitter
Interested in reaching a qualified audience of microelectronics industry decision-makers? Invest in host-read advertisements, and promote your company in upcoming episodes. Contact Françoise von Trapp to learn more.
Interested in becoming a sponsor of the 3D InCites Podcast? Check out our 2024 Media Kit. Learn more about the 3D InCites Community and how you can become more involved.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Francoise von Trapp (00:00):
This episode of the 3D InCites
podcast is sponsored by IMAPs,the premier global association
for microelectronics advancedpackaging enthusiasts.
A membership in IMAPs helpsyour company grow its advanced
packaging workforce throughprofessional education and
networking, advances your brand,and supports building
relationships.
IMAPS helps you learn, connect,and collaborate.
(00:20):
Learn more at imaps.org.
Hi there.
I'm Francoise von Trapp, andthis is the 3D InCites Podcast.
(00:42):
Hi everyone.
This week we're recording livefrom the IMAPS International
Symposium in San Diego, andwe're talking about, as always,
advancements in the advancedpackaging space or electronic
packaging or microelectronics.
And today, Subu Iyer, Dr. ofUCLA gave a keynote talk titled
(01:05):
Strategic Directions forElectronics Packaging.
And Dr.
Iyer was formerly with IBM.
He's also been with the NAPMP.
He's been back and forth toUCLA.
And he is the very first personI ever heard present on the
concept of disaggregating systemon chip technology into its
functional parts andre-aggregating them, which thus
(01:28):
became what he calls dielets,which everybody else calls
chiplets.
And I noticed from this prepresentation that you're now
calling them chiplets.
Subu Iyer (01:36):
Or dielet.
Francoise von Trapp (01:37):
Or dielets.
So welcome to the podcast,Subu.
Subu Iyer (01:41):
It's great, uh Francoise.
I'm uh so happy to be here.
I I've been giving this podcastuh a few times, and uh and I
also noticed that it is verypopular, and I listen to many of
your podcasts as well.
Francoise von Trapp (01:53):
Well, I appreciate that.
I'm so glad.
You know, I've been part ofthis industry for 20 years, and
I think I first heard youprobably when I started 3D
Insights in 2009.
You know, I've always lovedyour talks because you are a
professor and you know how tohold a crowd, and you were
actually saying today thatsomeone told you don't take your
(02:14):
knowledge giving seriously,that you really have to be an
entertainer.
Subu Iyer (02:18):
That is correct.
Francoise von Trapp (02:20):
So and I've met your students.
Remember the time we haddinner?
Subu Iyer (02:23):
Yeah, yeah.
Francoise von Trapp (02:24):
And I met all your students.
Absolutely.
You won a Research Institute ofthe Year award one year for
your work in UCLA TV.
Subu Iyer (02:30):
From from CD3 Insights, yeah.
Francoise von Trapp (02:32):
Yeah.
Subu Iyer (02:33):
So And that was uh right after we uh very very soon
after we started our withinabout three or four years of our
uh I came to UCLA.
So that was actually a big dealand meant a lot to us and to
our students.
Thank you so much.
Francoise von Trapp (02:46):
You're welcome.
I loved your talk this morning.
I want to hit on some of thehighlights.
Subu went through pretty muchthe history of advanced
packaging, and we're not wedon't have time to do that here.
But one of the things you saidis there are no chiplets.
So what did you mean by that?
Subu Iyer (03:03):
Yeah, I think the concept of chiplets, you talked
about like pioneering this wholethought process of um
disaggregation, right?
And the key element of thedisaggregation is a chiplet or a
dialet.
And actually the two are meanslightly different things, okay?
A chiplet is actually a designconstruct.
(03:24):
A dialet is a hard piece ofsilicon, all right?
And so when we integrate it ona substrate, it's a dilet.
When we design it, okay, it's achiplet.
Francoise von Trapp (03:37):
Okay, that's the first time I've heard
that.
Subu Iyer (03:39):
Yeah, so it's it's the same difference between a
chip and a die, right?
Francoise von Trapp (03:43):
Right.
Subu Iyer (03:44):
A designer designs a chip.
Francoise von Trapp (03:45):
Right.
Subu Iyer (03:46):
He sends it off the data to the foundry, the foundry
then makes a wafer, the wafersthen cut up into dyes.
Right?
We don't cut up the into chips,we cut up into dies.
The dyes are assembled, andthen they again mysteriously
become a chip.
Francoise von Trapp (04:01):
Oh my god.
You just blew my mind.
That is again the first this iswhat I love about you.
You know, that you you explainthings so well for people who
are not really technical.
And I s imagine this is becausethis is how you address your
freshman class.
Subu Iyer (04:17):
And my management.
Francoise von Trapp (04:18):
And your management But it's good because
somebody needs to be able to dothat.
You know, there's no claritythat we're looking for it.
Subu Iyer (04:25):
Oh, let me go back to this question uh which you
asked, which we kind of gotsidetracked on, which is I said
there were no chiplets.
Francoise von Trapp (04:32):
Right, okay.
Subu Iyer (04:33):
Now that has to be taken in context, okay?
There are chiplets.
There is no triplet ecosystemthat is universal.
Okay.
In other words, let's say, youknow, company A produces
chiplets that they use in theirown systems.
All right.
Company B produces chipletsthat they use in their own
systems.
And the company B tripletsdon't work with company A and
(04:54):
vice versa.
Right?
So when I say there is nochiplet ecosystem, it what it
means is that if we had astudent, for example, who gets
up one morning with a brilliantidea and says, Hey, I want to
use this company A chiplet and Iwant to integrate it with
company B triplet and get someother chiplets from somewhere
else and put it all together andbuild this really unique
system, he cannot do that.
(05:15):
Okay, he cannot do that becausethese two chiplets from these
two different companies and anyother company do not talk to
each other.
And uh they have differentmechanical standards, they're
different sizes, and they don'tlike fit together nicely.
All right.
Francoise von Trapp (05:30):
Because they weren't designed to fit.
Subu Iyer (05:32):
They were not designed, they were designed to
work in their own ecosystem.
Francoise von Trapp (05:36):
So isn't that the difference between a
system and a package whereyou're taking different
functionality chips and justinterconnecting them closer
together?
And because you said before thechiplet is a function of
design.
Right.
Correct.
One of the things the industryhas been working on is that
interface, right?
Subu Iyer (05:54):
The UCIER Yeah, there have been many interfaces that
have come and gone.
Francoise von Trapp (05:58):
But okay, so what you're working on at
UCLA then, too, are youpromoting this ecosystem?
Subu Iyer (06:06):
We are a university, right?
And so our goal is not someproprietary system, right?
We want to kind of dispenseknowledge, we want everybody to
be able to work with each other,and most importantly, we want
to encourage rather than stifleinnovation.
So today, right, if you look atthis, right, the number of
people who are buildingintegrated systems, right, from
(06:28):
scratch are very limited.
Okay, and that's because, youknow, first of all, the chiplets
are not available.
Secondly, nobody's going totalk to you if uh unless you are
willing to like to make a lotof parts, right?
I mean, if you're thinkingabout something like doing a
prototype or building a smallnumber of parts and so on,
customizing for like a specificapplication, you're out of luck.
(06:51):
Okay, nobody will even talk toyou.
However, okay, if you have aplatform that is universal,
think about Lego blocks, right?
So, Lego blocks, you havedifferent kinds of Lego blocks,
but they all have onecharacteristic they fit on each
other, right?
Right?
They may have different colorsand different sizes and whatnot,
but they all fit together.
Francoise von Trapp (07:12):
They also have some different shapes.
Subu Iyer (07:14):
They have different shapes and so on and so forth,
right?
Now, we probably don't want tohave chips of different shapes,
okay, but you know, the pointhere is if you have like uh
standardized sort of dimensions,right?
X dimensions and y dimensions,and you have standardized
mechanical characteristics,standardized bump locations, and
(07:36):
so on and so forth, then everychip can talk to every chip,
okay, and uh standardizedprotocol, right?
And the protocol and whathappens, and the thing that I
was trying to emphasize todaywas we can actually discard
protocols once we make thedimensions of the bump pitches
really, really small, and reallyas small as like sub 10
microns, which is possibletoday.
(07:58):
So without sort of if you justlet this thing flow like it has
been flowing, we will within thenext one or two years reach a
point where say people say, hey,we don't need UCIE, we don't
need this, we don't need that.
We can just use the nativeinterface between them between
these chips and continue as ifit was the same chip.
Francoise von Trapp (08:20):
Okay.
Subu Iyer (08:20):
Right?
Because now there is nodifference between communicating
on one chip and communicatingbetween chips.
Francoise von Trapp (08:27):
So is that what you mean by packaging being
an extension of monolithicchips?
Subu Iyer (08:31):
Exactly.
So like when you design amonolithic chip, right, we have
these IP blocks.
And these IP blocks aredesigned by various people,
okay, and they're available ascode, right?
Basically.
They have things likedescriptors, like a VHDL
descriptor and a.lef descriptor,which is uh talks about the
(08:52):
physical locations and so on ofwhere the pins are for that
particular IP block, and thenyou synthesize a chip with that,
with those IP blocks, right?
And you're downloading you'redownloading code to actually
integrate this thing.
So now when we transition tolike the chiplet or dialet
regime, you're no longer dialetdownloading code, you're
(09:13):
actually downloading real harddies.
But if these dies can bedescribed the same way, like
with these abstractions, theycall abstractions, and these
abstractions are like functionaldescriptions of the chips
saying, hey, look, pin numberone does this, pin number two
does that, and so on.
And these pins can communicatewith pin number A on the other
(09:36):
side, pin number B on the otherside, and basically all that you
need is to connect them with awire and life goes on as normal,
then you have achieved an openecosystem for these chips.
Francoise von Trapp (09:47):
So your prediction is that we won't need
the UCIE at some point.
Subu Iyer (09:53):
Correct.
We won't need the UCIE.
I mean you can still use UCIEand whatever things you want,
but they're gonna come withtheir own overheads and you can
achieve the same result evenwithout using it.
Francoise von Trapp (10:05):
Right.
Okay.
Subu Iyer (10:05):
But if you have something that already has UCIE,
you can still use it.
Francoise von Trapp (10:11):
Okay.
One of the other things yousaid that stuck with me, um, not
on the chiplet topic, but maybeon the chiplet topic actually,
is that the change in emphasisfor what packaging is has gone
from being the protection of thedevice to power.
Subu Iyer (10:30):
Yeah, power delivery.
Communication and cooling,right.
Francoise von Trapp (10:34):
And this is all to address this terrible
power-hungry data centeractivity.
I love that you use the exampleof doing a query for what
Taylor Swift is wearing todayand generating how much power
that causes.
People are out there, thegeneral public, don't understand
these frivolous uses and howmuch energy it takes.
Subu Iyer (10:58):
So I think, you know, one has to be careful, right?
I mean, think about how we didthis in the old days.
Like I would have to walk tothe library or take a bus to the
library or w or some some sortof mode of transportation, then
I would go check in, then Iwould have to go search for the
books, then I would have to openthe books and find the page and
(11:18):
and all that stuff, right?
Now that too takes energy.
Francoise von Trapp (11:23):
Yeah.
Subu Iyer (11:23):
All right.
Yeah.
And the amount of energy I needto do that, right, just driving
to the library, right?
Right.
Is a lot more than this uhenergy, right?
So you have to look at it fromthat perspective.
Right.
Okay, but I think what happensnow is because this is so easy,
okay, we use it in trivial ways.
(11:43):
Like, you know, how does itmatter what she's wearing?
Okay.
Right, right.
Francoise von Trapp (11:48):
Well, that's what I mean.
I mean, it's one thing to useit for research.
Subu Iyer (11:52):
Yeah, I think, right?
Uh, overall, we're all verycurious people in general,
right?
We will continue to use this.
If you can do it, you will doit.
Right.
Right?
So the question now is not howdo I restrict the use and say,
if somebody asks uh what isstalysting, I'm not gonna
answer.
Or Gemini is not gonna answer.
That is not the right answer.
(12:13):
So the right answer is to say,how do I make the energy
required to answer that query aslow as possible?
And that is where I thinkadvanced packaging plays a big
role.
Francoise von Trapp (12:23):
Well, and that's what our industry can do,
right?
It can solve it from theperspective of let's just make
it require less energy versusrestrict people from using it
frivolously and can contributingto global warming, right?
Exactly.
Do you think that's one of thereasons that there is a bigger
focus by the front-end guys arestarting to realize that
advanced packaging is about morethan just protection of chips?
Subu Iyer (12:47):
Yeah, I think so.
I think there's a better andbetter understanding and I of
what the value that packagingbrings, right?
The value that packaging bringsis far greater than just like
uh packaging a chip like in thegood old days.
Now it is actually delivering asystem.
Right.
Right?
And so the value addedpackaging now is significantly
(13:10):
in a different dimension than itused to be.
Right.
And there, and that's thereason I said, right, I mean,
and a few people came to meafter the talk.
I love it.
We you know we're sick andtired of this 1.5%, 3.5% gross
margin.
We own also 50%.
Francoise von Trapp (13:25):
Right.
Subu Iyer (13:25):
And TSMC figured that out and said, okay, you know
what?
We're gonna actually have uhthis advanced packaging, it's
gonna be like out of this world,and we're gonna get 50% gross
margin.
So they are the TSMC is notdoing this in because of
philanthropy.
Francoise von Trapp (13:43):
Right.
Subu Iyer (13:43):
Okay.
Francoise von Trapp (13:44):
I mean No, they're a business, they want to
make money, I get it.
Subu Iyer (13:47):
And I think, you know, I think what the packaging
guys should do is they have toget out of this mindset that I
have to make everything cheap,you know, I can't use advanced
uh technology because that'sexpensive.
You have to ask yourself, whatis the value that I'm bringing?
Right.
And if that value can justifythe cost, which clearly does,
(14:10):
right, uh, I think there's amarket.
Francoise von Trapp (14:12):
Right.
Yeah, I completely agree.
I've been trying to say thatfor a long time.
You know, I'm not arrogantenough to think that my belief
in this and my evangelizing foryears is that I'm not sure.
I don't have that problem.
You know, for the well, no, butyou are scientist, you're you
are you are an engineer, youunderstand this fundamentally,
and you have made this happenover the years.
I've only been the voice of allof these thoughts, and I
(14:38):
believe it, and I'm reallyexcited to have contributed to
that just by getting the messageout there because I think that
you know, messaging it takes awhile.
But, you know, I saw this, Iunderstood this a long time ago,
but realized that the theindustry wasn't catching on in
(14:58):
the sense that it takes so longin this industry for people to
realize something that to meseemed really um, you know, you
can't see the forest for thetrees kind of thing.
Subu Iyer (15:09):
When you're in the weeds, you can't see anything.
Francoise von Trapp (15:11):
Right, and so it was kind of my job to say
it over and over and over again.
Subu Iyer (15:15):
No, absolutely.
And I think, you know, I'llI'll say this like um uh I
remember when I was IBM and Italked to some of my colleagues,
very experienced people, very,very respected people, and I
said, hey, look, you know, howwhat do we need to do to make
the bump pitch smaller?
And uh a very, very uhrespected guy comes and tells
me, Hey, and he was an IBMfellow and all that stuff, okay?
(15:36):
So very, very well known guy.
He comes to me and says, Youknow, Subhu, you're full of, you
know, this.
First of all, it's verydifficult to do, but even if you
could do it, nobody needs it.
We've been doing it this wayfor the last 40 years, and there
is no need to change.
Francoise von Trapp (15:53):
And that is the mantra of this industry.
We change, we don't changeanything until there's a need to
change it, until it doesn'twork anymore, and then we have
to develop something new.
Subu Iyer (16:03):
Yeah, and another sort of thing that came about
along the same lines is likeroadmaps.
So when you're in the siliconbusiness, I come from the
silicon end of things, right?
So I I went and I suddenlybecame the uh director of
packaging, and that's a funnystory itself.
Okay, uh, I actually was I hada small group and was working
with a company called Micron,uh, which was um uh sort of
(16:25):
building this 3D stack memory,and they came to us to work with
us, and so you know, I wastrying to get all this done
done, TSVs and all this goodstuff, right?
And I was having such a hardtime with the packaging guys,
the classical packaging guys.
They said, Who wants this?
This is never gonna work, blah,blah, blah, and nobody needs
it.
And so I would go to my boss,who is a guy called Gary Patton,
(16:46):
and I would say, Gary, youknow, somehow a lot of problems.
One fine day, the guy, youknow, Gary just blew up and he
said, You know, so I'm tired ofyou complaining about these
packaging guys.
So from tomorrow, you're gonnamanage them.
Francoise von Trapp (16:59):
There you go.
Subu Iyer (17:00):
Yeah, so so all of a sudden, right, I have this
300-plus uh group of packagingengineers, and now you know they
kind of have to do what I tellthem to do, right?
And so now it is a question ofgetting education across and
getting them to understand whatwe're trying to do and so on and
so forth.
And I think one of the things Iasked them when I started this
(17:23):
job was, hey, uh can you show meyour packaging roadmap?
So what they said was very,very enlightening, and I think
this is a very important thingto remember.
And they said, Subu, we don'twork with roadmaps.
Okay, because when you're insilicon, you know the roadmap is
Moore's law, right?
Every year we know what to do.
(17:44):
So, but they said we don't dothis.
The chip guys come to us with aproblem and we solve it.
And so I said, and the nextnext guy, the next chip comes,
we solve that too.
And is that the same solution?
No.
So I you know, and so I said,look, the main thing we have to
do here is we have to have astructured roadmap so that we
(18:04):
can tell the chip guys what isgonna come about, okay?
And they can plan for it, andyou can plan for it, and this
makes everything smoother.
So I think you know that isstill not happened in the even
with the heterogeneous.
Yeah, even with theheterogeneous T you know, TSMC
does it one way, these uh theguys do it this way, and you'll
(18:25):
see that like the OSATs, forexample, are trying to get into
the business, but everybody hasa different um different
roadmap.
Yeah, okay, and these roadmapsdon't talk to each other, they
don't flow from one to theother, and so on and so forth.
Now, and the reason why TSMC iskind of successful is their
roadmap is self-consistentwithin their ecosystem.
Francoise von Trapp (18:47):
Right.
Subu Iyer (18:47):
And that is that works for sure, as long as
you're willing to go theretoday.
Francoise von Trapp (18:52):
But it's also a problem because you can't
have one company dominatedominate because if something
happens to that company, we'rescrewed.
Subu Iyer (19:00):
Yeah, and it's also it's also not good for
innovation, right?
I mean, yeah.
So I think roadmapping is good,and that's one of the reasons,
right?
We started this uh uh effortwhich is funded by the
government actually calledMRHIEP, the manufacturing
roadmap for heterogeneousintegration and electronics
packaging.
And that roadmap is actually onour website.
Anybody can go and look at it.
(19:20):
And we uh when when I was inthe NAPMP, that was the roadmap
we followed.
Francoise von Trapp (19:25):
Okay.
Subu Iyer (19:26):
Okay, and uh that is uh the roadmap I think the
industry should follow.
Francoise von Trapp (19:29):
Did you ever see Back to the Future?
Yeah, long ago.
Yeah.
Do you remember Doc says wherewe're going, we don't need
roads.
Yes.
Okay, so since you brought upthe NAPMP.
Subu Iyer (19:42):
Oh, okay, I and we don't have too much time left.
Francoise von Trapp (19:45):
I did want to get there, and you brought us
there, so I'm happy about that.
You said today you showed aslide that talked about the
NAPMP structure and the sixhardware eco and ecosystem
thrusts and piloting facilityand prototyping challenges, and
then you had all these questionmarks across it because all of
this because of the currentadministration has scrapped the
NAPMP, but you told people don'ttoss this work, it's still
(20:09):
important.
Can you talk about that alittle bit?
Subu Iyer (20:11):
Correct.
First of all, let's be clear.
I don't think the governmenthas tossed NAP and P.
They are redefining it andprobably rebranding it.
And every new administrationwants to do that, right?
I mean, uh they want to kind ofmake sure that it reflects
their sort of ideas of what isuh what is important and so on.
And and that's perfectly okay.
(20:32):
I don't have any issue withthat.
I I think the new uh BAA that Ihave seen is broad enough and
includes uh in a very broadsense, you know, heterogeneous
integration, packaging, assemblytests, and all this stuff, you
know, which are which are thewhat these these five areas used
to address.
So I think, you know, myfeeling, okay, and I mean time
(20:55):
only will tell, is that theideas of the original thing are
still sort of valid and fitnicely actually into the new
structure.
And if you look at wherethey're pushing, right, the the
the new BAA is pushing AI.
Okay, it's pushing a wholebunch of other things.
But the AI push, okay, isclearly something that will be
(21:20):
driven by advanced packaging.
And you're not gonna get thoseresults, okay, if you want to
reduce the power and all thatgood stuff.
You're not gonna get thatwithout, you know, uh basically
following the kind of structureand the ideas that I presented
that I developed when I was inthe NAP and P and also what I
(21:41):
sort of briefly presented heretoday.
You have to address thesubstrate issue, you have to
address the equipment newequipment that is needed for
this, you have to developprocesses for it, you have to
address thermal and powerdelivery, you have to address
connectors, and you have toaddress the chip right problem,
and you have to address the EDAproblem.
(22:02):
Okay, those key elements arekey are still very important.
And you also need, I mean, it'snot enough to actually just
build these in abstract.
Okay, you need a place to buildthese.
Okay, and if you don't have aplace to build this, you'll have
to go to China, you'll have togo to Taiwan, you'll have to go
somewhere else to do it.
And so it is really, really,really, really important that we
(22:25):
provide a safe place where uhinnovators in this country, and
we have the best innovators inthis country, can actually sort
of try out their ideas withadvanced packaging and be able
to get parts back, test them,make sure they work, and that
will lead to bigger and biggervolumes and manufacturing in the
(22:46):
US.
So I firmly believe, okay, thatthe strategy that we put
together with my colleagues whenI was in the NAPMP, and what I
kind of re-enunciated today,okay, are vital for making the
country competitive, okay, in inadvanced packaging.
(23:09):
And one thing is very clearnobody, no country, no uh
country has a monopoly oninnovation.
Right.
Okay.
And today, right, I would sayhalf my students come from
outside this country, right?
And uh it's more and moredifficult nowadays with all
these restrictions for these forus to accept students from
(23:30):
certain countries.
And this is going to be a veryserious problem.
Big, big one the way we've beeninnovative is we've been able
to attract the best talent.
Francoise von Trapp (23:39):
Are the H1B visas impacting students?
Not really.
Subu Iyer (23:43):
They don't they don't impact students, okay, but they
do impact the ability of thesestudents to get jobs.
And if they don't get jobs andthey don't get these visas,
okay, they're going to go backto their home country.
So here we are, we put a lot ofeffort educating them, they
become fantastic engineers, andwe say go back to your country.
Right?
And um and there are jobs inthose countries, okay?
Francoise von Trapp (24:07):
And isn't it getting to the point where
there's other places they canget their education besides the
US as well?
Subu Iyer (24:12):
That is also a big, big danger.
One of the big big uh sort ofexports we have is actually
higher education.
Okay.
We have easily one of the besthigher education uh
infrastructures in the world.
And uh if we kind of dismantleit, right, and we are
dismantling it in many, manyways, I'm not saying that the
universities are not to blame.
(24:33):
The universities have actuallybrought a lot of these problems
on themselves by sort of notfocusing on the real problems.
So, for example, right, and andpart of that is actually the
engineering faculty who have notparticipated, and the science
faculty and the medical facultywho have not participated in the
(24:56):
administration of theuniversity.
So if you look at theadministration of the
university, it is dominated bythe humanities, the
philosophers, the sociologists,and so on and so forth.
And they have a differentagenda, okay, which is a very
idealized agenda, which is verydifficult to actually implement,
right?
Like if you think about likefree buses and free food and
free groceries for all.
(25:17):
I mean, that's not happening.
You cannot sustain that.
Right.
But the people who have to sortof uh put uh the voice of
reason here are notparticipating.
And so the universities havesort of swung that pendulum to
the other side, but the reactionis also not correct because by
withholding funding touniversities, those people don't
get government funding.
(25:38):
Okay, they don't have researchprojects, their overhead is very
low.
So they have plenty of time totalk about these problems.
The guys who get affected arepeople like me, okay, who
basically say cannot fundresearch.
It costs me $100,000 to fund aPhD student per year.
Wow.
Yeah.
Yeah.
So and you know, and for anexperimental number uh PhD,
(26:01):
right, which is what we need toget people skilled in
manufacturing, that number isalmost twice.
So I computed that it takesabout a million bucks to get a
PhD out of the door.
Wow.
No, we spend a million bucksand we say go back.
We're not gonna give you a job.
This is complete nonsense.
Okay, and unfortunately, right,the people who are in power
(26:23):
here do not seem to appreciatethat.
You cannot get a guy fromWalmart to be like a top-notch
Sundar Pichai at Google.
All right?
Francoise von Trapp (26:33):
Yes.
Subu Iyer (26:33):
I mean, and no arguments from me.
Yeah, and this is the problem.
Yeah, we're we're educatingthese people, we need to
benefit.
Right.
Okay.
I'll stop.
Francoise von Trapp (26:43):
No, I mean, I would talk to you all day
about this, but I can't.
So um I am just so happy tohave had this opportunity to
talk to you one more time.
I've really enjoyed ourconversations over the years, so
thank you for joining me today.
Subu Iyer (26:56):
Yeah, and it's been a pleasure talking with you and
uh having this free free-rollingconversation.
Okay.
Francoise von Trapp (27:02):
And then people get to listen to it.
So that's the fun.
I mean part.
All right.
Okay, thank you so much.
Thanks, Subu.
Subu Iyer (27:09):
Bye.
Francoise von Trapp (27:13):
For several years at the IMAP Symposium,
we've been following theevolution of co-packaged optics
as they make their way tocommercialization.
Now, this year, Jan Varnemanfocused her Wednesday evening
panel on the topic.
The topic was building theecosystem for co-packaged
optics.
And joining me to talk aboutthis are three of the panelists,
and they're all from our membercompanies.
(27:34):
I have Mike Kelly of Amcor, TolLiteken of Fraunhofer IZM, and
John Knickerbacher of IBM.
Welcome to the podcast, guys.
John Knickerbocker (27:43):
Thank you.
Thank you.
Francoise von Trapp (27:45):
Before we dive into the topic of the day,
can you each just tell us alittle bit about what you do at
your companies and how you'vebecome involved in co-packaged
optics, why you're interested init?
Mike Kelly (27:55):
Uh hi, my name is Mike Kelly.
The question is how do I gotinvolved in Silicon Photonics?
So I think like all of theprojects that we undertake, it's
something that our customersare interested in, and we need
to get prepared.
So uh Silicon Photonics, aseveryone knows, has been around
for a long, long time.
We've worked on it on and offfor you know years, I would say
(28:20):
around a decade probably.
What's completely aggravatedthe situation is all this
bandwidth requirements in thedata center for AI, and we're
looking to help our customershandle that in uh in a better
way than what what is possibletoday, staying completely
electrical.
So I think that's kind of thebackground on it, and uh, you
(28:40):
know, we're looking forward tomaking progress.
Francoise von Trapp (28:44):
Okay.
Tolga?
Togla Tekin (28:46):
Hi, my name is Tolga Tolga Tekken.
I'm in charge of the photonicplasmonic systems group in front
of RyCM.
Background optical signalprocessing.
I get introduced to photonicsin back to 95 since then working
on photonics, plus the onpackaging side.
So what we observe is actuallythe main bottleneck, off-chip,
(29:11):
off-core interconnects havinghigh bandwidth, low latency, and
high density interconnects.
So the promise is thatphotonics helping out with the
entire development coming fromtelecom, right?
John Knickerbocker (29:25):
That's right to leverage that.
Francoise von Trapp (29:27):
Okay, and John?
John Knickerbocker (29:29):
So uh maybe I'm a little different.
Uh my name is JohnNeckerbacher, and uh um I'm with
IBM Corporation, and I've beenuh developing advanced packaging
technologies for over fourdecades.
And uh and so uh a part ofthat, uh on and off through the
career has been uh various um uhtest vehicles that that uh
(29:51):
drove uh photonics and and uhoptical interconnect.
And over the last uh severalyears, uh I've I've led a team
that has uh Been driving some ofour um next generation
technologies, as we call it, uh,in terms of CPO technology and
advanced photonics.
And so, you know, I don'tactually consider myself a
photonics expert.
I'm not but I'm a packagingexpert, and we have many uh
(30:15):
photonics experts uh uh on ourteam.
And and so, you know, thecomplexity of photonics and and
co-packaged optics is such thatyou really need to have a broad
range of experts.
This technology is uh hascaught fire very recently, but
we started this more than adecade ago, back in the 2009
(30:36):
time period, and uh thesetechnologies kind of evolve from
a laboratory level, and uhultimately we want to go from
laboratory to prototyping andmanufacturing to help ourselves
and clients meet their needs.
And so that's how I've beeninvolved, uh helping to drive
this over the last severalyears.
Francoise von Trapp (30:55):
You know, as long as I've been in this
industry, which is 15 to 20years now, um when I started
hearing about photonics, fromwhat I understand, the issue has
always been on the packagingside.
You could do chip to chip.
The challenge was getting thesignal off the package.
You know, the conversation overthe years has ebbed and flowed
as developments happened andalso you know, drivers came
(31:19):
along, and it seems like nowwe've got this, as it always is
in the advanced packaging sideof things or in the industry
when there's a need for it, thatsuddenly there's this burst of
activity because now there's noother way to do it, or it's
going to solve a big industrychallenge.
So, just for the purposes ofour audience who may not
understand what co-packagedoptics are, we've got the
(31:40):
silicon photonics.
Um, what do we mean byco-packaged optics?
John Knickerbocker (31:46):
I've heard many different versions of what
people think co-packaged opticsis, and I I um uh looked and
found an ANSIS blog from uh2024, and I thought uh it very
well represented what uhco-packaged optics is, at least
from my vantage point.
This blog basically calls outthat CPO is uh an approach to
(32:09):
address the growing challengesaround bandwidth density,
communication latency, copperreach, and power efficiency uh
to support uh the nextgeneration optics and
electronics requirements.
To me, that really addressesthe key points for CPO.
Francoise von Trapp (32:26):
So basically, we're replacing the
copper with fiber optics in someway.
John Knickerbocker (32:32):
The reach of copper is challenging to go
beyond the one to two literrange, and optics is has that uh
opportunity to go that longerdistance and start to bring in
very high bandwidth and asenergy comes down to support
that in a power efficient way.
Mike Kelly (32:51):
I like your definition a lot.
I think that you know, whencoming at it from a strictly
packaging standpoint, when we'retalking about co-packaged
optics, it really gets down toyou know the provisions that
need to be inside the packagesomewhere to interface to the
the optical path.
And that's taken many formsover the last decade, but but it
(33:16):
seems like we've consolidatedon something now that makes
sense uh and is extensible.
And so now I think the the keyreally is making sure that we
have a a path to higher volumeso that it can be more or less
mainstream.
Seems like the the vision forthat is now nearer in than it
(33:36):
was ten years ago for sure.
So that's kind of what it meansto me.
Francoise von Trapp (33:40):
Okay.
Togla Tekin (33:40):
Indeed, the losses in ref transmission line
increase with the distance,right?
And for the time being, theelectro optical conversion takes
place at the edge of the panel.
So that's in the range oftwelve inches.
Right?
Would somehow corresponding to10, 12, 12 dB loss over there?
(34:03):
Around 60 gigabolt per second.
That that that's the metric,right?
So if it can come closer withelectro-optical conversion,
closer to the ASIC, whichrequires a higher bandwidth, so
then we'll that transmissionline will not consume so much
(34:24):
power.
Right?
It will be saving on that end.
That's that probably will yieldfour times minimum power
saving.
Francoise von Trapp (34:36):
So the driver for going to co-packaged
optics is primarily power?
Togla Tekin (34:42):
Basically the limitation of RF micro
transmission lanes.
Okay.
Francoise von Trapp (34:47):
There's a difference, right, between the
silicon photonics and theco-packaged optics.
Mike Kelly (34:52):
Power is definitely a part of it, but what the
industry needs is a lot morebandwidth for all these
disaggregated systems inparticular that are AI-centric.
Francoise von Trapp (35:02):
Right, okay.
Mike Kelly (35:03):
You've got to have more bandwidth for the next
generations, and this is the thebest way to achieve that.
Okay.
Togla Tekin (35:10):
I mean, let's refer to data center switch ASICs, 50
tera, 100 tera switches, right?
So the Redix on is limited.
It is SARDAS.
So that means for the timing,SARS speed is following the
bandwidth requirement of datacenter switches, right?
(35:31):
But each increase of SARDISspeed causes additional power.
So and then bringing all thosetogether requires a closer
integration of a OA conversioncloser to the switch ASIC
(35:53):
minimum.
Francoise von Trapp (35:55):
What's missing from the infrastructure
today that would make itpossible or that that's keeping
co-packaged optics from reachingcommercialization?
John Knickerbocker (36:07):
So I I think there's many obstacles to
really climb that curve and gofrom small volume research
demonstrators to volume that isused broadly across many
applications.
So I think one of the keydrivers now is AI, right?
(36:30):
And AI training and AIinference are very important.
And the bandwidth that's neededto support that effective
compute and especially theenergy being consumed to do even
just one training um algorithmcan take months at the present
(36:50):
time and significant energy tobring out these billion
parameter models algorithms thatsupport that information that
people want to access.
And the problem with that isthat's just the first turn on a
training module, and yet youoften need to go through more
(37:13):
iterations of fine tuning.
Uh likewise for inference uhoperation.
You also want to go throughsome iterations of fine tuning
and be efficient in this.
So as was stated earlier, I Ithink the value of CPO is both
in the bandwidth it can bring tothese applications that can
reduce the amount of time andenergy required for both
(37:35):
training and inference andfine-tuning, as well as the
ability to support theinterconnectivity of these uh
disaggregated systems across thedata center at longer distances
than can be done today withmany, many more GPUs and modules
that need to be connected inorder to be efficient.
Francoise von Trapp (37:54):
So, as far as a data center application,
for instance, a large languagemodel.
John Knickerbocker (37:59):
It's more about being um efficient with
energy and uh driving these verylarge language model algorithms
are just power hungry and needlots and lots of GPUs from you
know from hundreds to thousandsto tens of thousands.
And when you get at that scale,the uh the amount of energy
(38:20):
that's being consumed to createthese models and run them
through to be effective is justtoo large, right?
And so you can either reducethe algorithm size, right?
I think there's room for thatas well, or you can also uh
create this higher bandwidthconnectivity between these GPUs
so they can be uh more effectiveand get to the answer quick
(38:43):
more quickly.
And each is probably the rightway to go.
Uh some some combination.
Mike Kelly (38:48):
I think just a little fine-tune on that is like
you mentioned earlier, thealgorithms are so large.
Now they're trillions ofparameters, and so there's an
immensely greater amount of datamovement in the data center,
back and forth from thedifferent memory domains
surrounding all these GPUs.
So the amount of data thatneeds to go back and forth as
(39:11):
you train a model, or even whenyou infer from an existing
model, is orders of magnitudelarger now that LLMs have come
onto the scene.
This is still kind of attackingthe question why all the
interest in silicon photonicsnow?
Because the amount of data thatneeds to get moved around
between different GPUs andespecially between different
(39:34):
memory blocks in that GPU spaceis becoming and will become
larger than you can reallyhandle in copper, than you can
handle electrically.
It just we won't be able togrow the capability of data
centers if we stay electricallyconnected.
It's going to need to movebeyond that to get the amount of
(39:54):
bandwidth again to be able totackle these larger models.
Francoise von Trapp (39:59):
Okay, so we're understanding why the why
and what the driver is.
So, what's missing from theinfrastructure preventing us
from being able to adopt this?
Mike Kelly (40:09):
The thing that's been missing for a long time is
a really cost economic way tobuild silicon photonics
transceivers so that you can usethem ubiquitously everywhere.
That's work in progress still,it's not a done deal.
But a lot of the big umhardware companies that are
working on making this possiblehave put the stake in the
(40:31):
ground.
I have to have this.
And so, you know, I think it'senergized the the industrial
community.
Come around and say, we gottamake this happen now.
It's it's urgent, it's notoptional.
And to me, that's thedifference today than five years
ago or ten years ago.
Togla Tekin (40:48):
But so I would like to somehow differentiate,
right, the traditional datacenter are hard hyperscalers.
Francoise von Trapp (40:56):
Uh-huh.
Togla Tekin (40:57):
The CPO concept started over there.
With AI data center, sure, weneed high bandwidth inside the
data center, but additionallythe lowest latency in order to
have computing run parallel.
So I can probably give somenumbers on traditional data
centers, hyperscalers requiringthe acceptance of
(41:23):
electro-optical transceiversfrom globals is expected one
dollar per gigabit per second.
So if you can meet that target,you are in.
Otherwise you cannot be in thegame.
So possibly it will be similarin AI data centers, I could
(41:46):
imagine.
But since the requirements aredifferent, that several GPUs or
XPUs to be connected at the sametime.
It's more beyond than beyondthan the co-packaged optics.
So it's possibly in chipletlevel to be realized.
I mean, that's slightlydifferent.
(42:06):
I I will see more advancedthinker.
Francoise von Trapp (42:10):
Okay.
So we're gonna see firstimplementation of co-packaged
optics then in AI data centers.
Togla Tekin (42:18):
In traditional ones.
Francoise von Trapp (42:21):
You'll see it in hyperscalers first.
And when do we think when do wethink that's gonna happen?
Togla Tekin (42:29):
I mean if you meet one dollar per gigabit per
second today.
For the time being uhpluggables 400g is the right
current market.
Tomorrow 800 G.
So the vendors will be probablykeeping delivering those.
Mike Kelly (42:51):
Yeah, the pluggables you're talking about.
Yeah, you're right.
If you include pluggables intothis discussion, that's already
high volume, right?
So but when you're talkingabout co-packaged on in a large
GPU processor or CPU, that'sthat's different.
And that is still is still lowvolume.
Francoise von Trapp (43:17):
Okay.
So is it safe to say that wewill be having similar
conversations about this in fiveyears?
John Knickerbocker (43:26):
Or in five years so I think the the number
of optical connections permodule and and the bandwidth is
gonna climb over that timeperiod.
Yeah.
But I think uh it's gonna takeum, you know, pull from the
industry uh to you know climbthat curve.
(43:48):
So as any new technology goesfrom smaller volumes and
specialty, you start to get morestandards created and more
competition and more volume asthe uh the demand goes up and
and that drives itself in abusiness cycle to, you know,
(44:09):
winner is the one that givesthem a a cost-effective
solution.
And and so that's part of youknow, not just the technology,
but the business cycle tosupport a system requirements
that's cost effective and can bemet in the volumes of supply
chain that are needed.
And so all those things, likewhat's the exact timing?
(44:30):
Good question.
Um but I think over the nextfive years, if that's the time
horizon that that uh is beingdiscussed, that uh the the this
technology is gonna advancesubstantially in that time
period and beyond.
And so so again, I think you'regonna be talking um much higher
bandwidth, uh uh you knowpower, I think, um will come
(44:51):
down, but it may be secondary tostart with and then later
become a more important driver.
Uh so so again, I think allthese things are gonna advance,
and so it's actually veryexciting times uh because I see
the you know the the this topicarea is just caught fire in the
industry, and and so you've gotmany people working on it, and I
think it's gonna uh you know goat a faster pace.
Francoise von Trapp (45:13):
Well, then I think it's gonna be great to
continue having thisconversation and and following
this, I'm sure it'll be thetopic of many future IMAPs.
Well, thank you, all of you,for joining me today.
John Knickerbocker (45:26):
Thank you.
Thank you.
Francoise von Trapp (45:31):
Next time on the 3D Insights podcast,
we've got more from IMAPSymposium 2025.
You'll hear from one newcomerto the industry about why he is
interested in the field, andalso a group of students talk
about their experience at thisyear's IMAP.
There's lots more to come, sotune in next time to the 3D
Insights podcast.
The 3D Insights Podcast is aproduction of 3D Insights LLC.
This episode of the 3D InCites
podcast is sponsored by IMAPs,the premier global association
for microelectronics advancedpackaging enthusiasts.
A membership in IMAPs helpsyour company grow its advanced
packaging workforce throughprofessional education and
networking, advances your brand,and supports building
relationships.
IMAPS helps you learn, connect,and collaborate.
(00:20):
Learn more at imaps.org.
Hi there.
I'm Francoise von Trapp, andthis is the 3D InCites Podcast.
(00:42):
Hi everyone.
This week we're recording livefrom the IMAPS International
Symposium in San Diego, andwe're talking about, as always,
advancements in the advancedpackaging space or electronic
packaging or microelectronics.
And today, Subu Iyer, Dr. ofUCLA gave a keynote talk titled
(01:05):
Strategic Directions forElectronics Packaging.
And Dr.
Iyer was formerly with IBM.
He's also been with the NAPMP.
He's been back and forth toUCLA.
And he is the very first personI ever heard present on the
concept of disaggregating systemon chip technology into its
functional parts andre-aggregating them, which thus
(01:28):
became what he calls dielets,which everybody else calls
chiplets.
And I noticed from this prepresentation that you're now
calling them chiplets.
Subu Iyer (01:36):
Or dielet.
Francoise von Trapp (01:37):
Or dielets.
So welcome to the podcast,Subu.
Subu Iyer (01:41):
It's great, uh Francoise.
I'm uh so happy to be here.
I I've been giving this podcastuh a few times, and uh and I
also noticed that it is verypopular, and I listen to many of
your podcasts as well.
Francoise von Trapp (01:53):
Well, I appreciate that.
I'm so glad.
You know, I've been part ofthis industry for 20 years, and
I think I first heard youprobably when I started 3D
Insights in 2009.
You know, I've always lovedyour talks because you are a
professor and you know how tohold a crowd, and you were
actually saying today thatsomeone told you don't take your
(02:14):
knowledge giving seriously,that you really have to be an
entertainer.
Subu Iyer (02:18):
That is correct.
Francoise von Trapp (02:20):
So and I've met your students.
Remember the time we haddinner?
Subu Iyer (02:23):
Yeah, yeah.
Francoise von Trapp (02:24):
And I met all your students.
Absolutely.
You won a Research Institute ofthe Year award one year for
your work in UCLA TV.
Subu Iyer (02:30):
From from CD3 Insights, yeah.
Francoise von Trapp (02:32):
Yeah.
Subu Iyer (02:33):
So And that was uh right after we uh very very soon
after we started our withinabout three or four years of our
uh I came to UCLA.
So that was actually a big dealand meant a lot to us and to
our students.
Thank you so much.
Francoise von Trapp (02:46):
You're welcome.
I loved your talk this morning.
I want to hit on some of thehighlights.
Subu went through pretty muchthe history of advanced
packaging, and we're not wedon't have time to do that here.
But one of the things you saidis there are no chiplets.
So what did you mean by that?
Subu Iyer (03:03):
Yeah, I think the concept of chiplets, you talked
about like pioneering this wholethought process of um
disaggregation, right?
And the key element of thedisaggregation is a chiplet or a
dialet.
And actually the two are meanslightly different things, okay?
A chiplet is actually a designconstruct.
(03:24):
A dialet is a hard piece ofsilicon, all right?
And so when we integrate it ona substrate, it's a dilet.
When we design it, okay, it's achiplet.
Francoise von Trapp (03:37):
Okay, that's the first time I've heard
that.
Subu Iyer (03:39):
Yeah, so it's it's the same difference between a
chip and a die, right?
Francoise von Trapp (03:43):
Right.
Subu Iyer (03:44):
A designer designs a chip.
Francoise von Trapp (03:45):
Right.
Subu Iyer (03:46):
He sends it off the data to the foundry, the foundry
then makes a wafer, the wafersthen cut up into dyes.
Right?
We don't cut up the into chips,we cut up into dies.
The dyes are assembled, andthen they again mysteriously
become a chip.
Francoise von Trapp (04:01):
Oh my god.
You just blew my mind.
That is again the first this iswhat I love about you.
You know, that you you explainthings so well for people who
are not really technical.
And I s imagine this is becausethis is how you address your
freshman class.
Subu Iyer (04:17):
And my management.
Francoise von Trapp (04:18):
And your management But it's good because
somebody needs to be able to dothat.
You know, there's no claritythat we're looking for it.
Subu Iyer (04:25):
Oh, let me go back to this question uh which you
asked, which we kind of gotsidetracked on, which is I said
there were no chiplets.
Francoise von Trapp (04:32):
Right, okay.
Subu Iyer (04:33):
Now that has to be taken in context, okay?
There are chiplets.
There is no triplet ecosystemthat is universal.
Okay.
In other words, let's say, youknow, company A produces
chiplets that they use in theirown systems.
All right.
Company B produces chipletsthat they use in their own
systems.
And the company B tripletsdon't work with company A and
(04:54):
vice versa.
Right?
So when I say there is nochiplet ecosystem, it what it
means is that if we had astudent, for example, who gets
up one morning with a brilliantidea and says, Hey, I want to
use this company A chiplet and Iwant to integrate it with
company B triplet and get someother chiplets from somewhere
else and put it all together andbuild this really unique
system, he cannot do that.
(05:15):
Okay, he cannot do that becausethese two chiplets from these
two different companies and anyother company do not talk to
each other.
And uh they have differentmechanical standards, they're
different sizes, and they don'tlike fit together nicely.
All right.
Francoise von Trapp (05:30):
Because they weren't designed to fit.
Subu Iyer (05:32):
They were not designed, they were designed to
work in their own ecosystem.
Francoise von Trapp (05:36):
So isn't that the difference between a
system and a package whereyou're taking different
functionality chips and justinterconnecting them closer
together?
And because you said before thechiplet is a function of
design.
Right.
Correct.
One of the things the industryhas been working on is that
interface, right?
Subu Iyer (05:54):
The UCIER Yeah, there have been many interfaces that
have come and gone.
Francoise von Trapp (05:58):
But okay, so what you're working on at
UCLA then, too, are youpromoting this ecosystem?
Subu Iyer (06:06):
We are a university, right?
And so our goal is not someproprietary system, right?
We want to kind of dispenseknowledge, we want everybody to
be able to work with each other,and most importantly, we want
to encourage rather than stifleinnovation.
So today, right, if you look atthis, right, the number of
people who are buildingintegrated systems, right, from
(06:28):
scratch are very limited.
Okay, and that's because, youknow, first of all, the chiplets
are not available.
Secondly, nobody's going totalk to you if uh unless you are
willing to like to make a lotof parts, right?
I mean, if you're thinkingabout something like doing a
prototype or building a smallnumber of parts and so on,
customizing for like a specificapplication, you're out of luck.
(06:51):
Okay, nobody will even talk toyou.
However, okay, if you have aplatform that is universal,
think about Lego blocks, right?
So, Lego blocks, you havedifferent kinds of Lego blocks,
but they all have onecharacteristic they fit on each
other, right?
Right?
They may have different colorsand different sizes and whatnot,
but they all fit together.
Francoise von Trapp (07:12):
They also have some different shapes.
Subu Iyer (07:14):
They have different shapes and so on and so forth,
right?
Now, we probably don't want tohave chips of different shapes,
okay, but you know, the pointhere is if you have like uh
standardized sort of dimensions,right?
X dimensions and y dimensions,and you have standardized
mechanical characteristics,standardized bump locations, and
(07:36):
so on and so forth, then everychip can talk to every chip,
okay, and uh standardizedprotocol, right?
And the protocol and whathappens, and the thing that I
was trying to emphasize todaywas we can actually discard
protocols once we make thedimensions of the bump pitches
really, really small, and reallyas small as like sub 10
microns, which is possibletoday.
(07:58):
So without sort of if you justlet this thing flow like it has
been flowing, we will within thenext one or two years reach a
point where say people say, hey,we don't need UCIE, we don't
need this, we don't need that.
We can just use the nativeinterface between them between
these chips and continue as ifit was the same chip.
Francoise von Trapp (08:20):
Okay.
Subu Iyer (08:20):
Right?
Because now there is nodifference between communicating
on one chip and communicatingbetween chips.
Francoise von Trapp (08:27):
So is that what you mean by packaging being
an extension of monolithicchips?
Subu Iyer (08:31):
Exactly.
So like when you design amonolithic chip, right, we have
these IP blocks.
And these IP blocks aredesigned by various people,
okay, and they're available ascode, right?
Basically.
They have things likedescriptors, like a VHDL
descriptor and a.lef descriptor,which is uh talks about the
(08:52):
physical locations and so on ofwhere the pins are for that
particular IP block, and thenyou synthesize a chip with that,
with those IP blocks, right?
And you're downloading you'redownloading code to actually
integrate this thing.
So now when we transition tolike the chiplet or dialet
regime, you're no longer dialetdownloading code, you're
(09:13):
actually downloading real harddies.
But if these dies can bedescribed the same way, like
with these abstractions, theycall abstractions, and these
abstractions are like functionaldescriptions of the chips
saying, hey, look, pin numberone does this, pin number two
does that, and so on.
And these pins can communicatewith pin number A on the other
(09:36):
side, pin number B on the otherside, and basically all that you
need is to connect them with awire and life goes on as normal,
then you have achieved an openecosystem for these chips.
Francoise von Trapp (09:47):
So your prediction is that we won't need
the UCIE at some point.
Subu Iyer (09:53):
Correct.
We won't need the UCIE.
I mean you can still use UCIEand whatever things you want,
but they're gonna come withtheir own overheads and you can
achieve the same result evenwithout using it.
Francoise von Trapp (10:05):
Right.
Okay.
Subu Iyer (10:05):
But if you have something that already has UCIE,
you can still use it.
Francoise von Trapp (10:11):
Okay.
One of the other things yousaid that stuck with me, um, not
on the chiplet topic, but maybeon the chiplet topic actually,
is that the change in emphasisfor what packaging is has gone
from being the protection of thedevice to power.
Subu Iyer (10:30):
Yeah, power delivery.
Communication and cooling,right.
Francoise von Trapp (10:34):
And this is all to address this terrible
power-hungry data centeractivity.
I love that you use the exampleof doing a query for what
Taylor Swift is wearing todayand generating how much power
that causes.
People are out there, thegeneral public, don't understand
these frivolous uses and howmuch energy it takes.
Subu Iyer (10:58):
So I think, you know, one has to be careful, right?
I mean, think about how we didthis in the old days.
Like I would have to walk tothe library or take a bus to the
library or w or some some sortof mode of transportation, then
I would go check in, then Iwould have to go search for the
books, then I would have to openthe books and find the page and
(11:18):
and all that stuff, right?
Now that too takes energy.
Francoise von Trapp (11:23):
Yeah.
Subu Iyer (11:23):
All right.
Yeah.
And the amount of energy I needto do that, right, just driving
to the library, right?
Right.
Is a lot more than this uhenergy, right?
So you have to look at it fromthat perspective.
Right.
Okay, but I think what happensnow is because this is so easy,
okay, we use it in trivial ways.
(11:43):
Like, you know, how does itmatter what she's wearing?
Okay.
Right, right.
Francoise von Trapp (11:48):
Well, that's what I mean.
I mean, it's one thing to useit for research.
Subu Iyer (11:52):
Yeah, I think, right?
Uh, overall, we're all verycurious people in general,
right?
We will continue to use this.
If you can do it, you will doit.
Right.
Right?
So the question now is not howdo I restrict the use and say,
if somebody asks uh what isstalysting, I'm not gonna
answer.
Or Gemini is not gonna answer.
That is not the right answer.
(12:13):
So the right answer is to say,how do I make the energy
required to answer that query aslow as possible?
And that is where I thinkadvanced packaging plays a big
role.
Francoise von Trapp (12:23):
Well, and that's what our industry can do,
right?
It can solve it from theperspective of let's just make
it require less energy versusrestrict people from using it
frivolously and can contributingto global warming, right?
Exactly.
Do you think that's one of thereasons that there is a bigger
focus by the front-end guys arestarting to realize that
advanced packaging is about morethan just protection of chips?
Subu Iyer (12:47):
Yeah, I think so.
I think there's a better andbetter understanding and I of
what the value that packagingbrings, right?
The value that packaging bringsis far greater than just like
uh packaging a chip like in thegood old days.
Now it is actually delivering asystem.
Right.
Right?
And so the value addedpackaging now is significantly
(13:10):
in a different dimension than itused to be.
Right.
And there, and that's thereason I said, right, I mean,
and a few people came to meafter the talk.
I love it.
We you know we're sick andtired of this 1.5%, 3.5% gross
margin.
We own also 50%.
Francoise von Trapp (13:25):
Right.
Subu Iyer (13:25):
And TSMC figured that out and said, okay, you know
what?
We're gonna actually have uhthis advanced packaging, it's
gonna be like out of this world,and we're gonna get 50% gross
margin.
So they are the TSMC is notdoing this in because of
philanthropy.
Francoise von Trapp (13:43):
Right.
Subu Iyer (13:43):
Okay.
Francoise von Trapp (13:44):
I mean No, they're a business, they want to
make money, I get it.
Subu Iyer (13:47):
And I think, you know, I think what the packaging
guys should do is they have toget out of this mindset that I
have to make everything cheap,you know, I can't use advanced
uh technology because that'sexpensive.
You have to ask yourself, whatis the value that I'm bringing?
Right.
And if that value can justifythe cost, which clearly does,
(14:10):
right, uh, I think there's amarket.
Francoise von Trapp (14:12):
Right.
Yeah, I completely agree.
I've been trying to say thatfor a long time.
You know, I'm not arrogantenough to think that my belief
in this and my evangelizing foryears is that I'm not sure.
I don't have that problem.
You know, for the well, no, butyou are scientist, you're you
are you are an engineer, youunderstand this fundamentally,
and you have made this happenover the years.
I've only been the voice of allof these thoughts, and I
(14:38):
believe it, and I'm reallyexcited to have contributed to
that just by getting the messageout there because I think that
you know, messaging it takes awhile.
But, you know, I saw this, Iunderstood this a long time ago,
but realized that the theindustry wasn't catching on in
(14:58):
the sense that it takes so longin this industry for people to
realize something that to meseemed really um, you know, you
can't see the forest for thetrees kind of thing.
Subu Iyer (15:09):
When you're in the weeds, you can't see anything.
Francoise von Trapp (15:11):
Right, and so it was kind of my job to say
it over and over and over again.
Subu Iyer (15:15):
No, absolutely.
And I think, you know, I'llI'll say this like um uh I
remember when I was IBM and Italked to some of my colleagues,
very experienced people, very,very respected people, and I
said, hey, look, you know, howwhat do we need to do to make
the bump pitch smaller?
And uh a very, very uhrespected guy comes and tells
me, Hey, and he was an IBMfellow and all that stuff, okay?
(15:36):
So very, very well known guy.
He comes to me and says, Youknow, Subhu, you're full of, you
know, this.
First of all, it's verydifficult to do, but even if you
could do it, nobody needs it.
We've been doing it this wayfor the last 40 years, and there
is no need to change.
Francoise von Trapp (15:53):
And that is the mantra of this industry.
We change, we don't changeanything until there's a need to
change it, until it doesn'twork anymore, and then we have
to develop something new.
Subu Iyer (16:03):
Yeah, and another sort of thing that came about
along the same lines is likeroadmaps.
So when you're in the siliconbusiness, I come from the
silicon end of things, right?
So I I went and I suddenlybecame the uh director of
packaging, and that's a funnystory itself.
Okay, uh, I actually was I hada small group and was working
with a company called Micron,uh, which was um uh sort of
(16:25):
building this 3D stack memory,and they came to us to work with
us, and so you know, I wastrying to get all this done
done, TSVs and all this goodstuff, right?
And I was having such a hardtime with the packaging guys,
the classical packaging guys.
They said, Who wants this?
This is never gonna work, blah,blah, blah, and nobody needs
it.
And so I would go to my boss,who is a guy called Gary Patton,
(16:46):
and I would say, Gary, youknow, somehow a lot of problems.
One fine day, the guy, youknow, Gary just blew up and he
said, You know, so I'm tired ofyou complaining about these
packaging guys.
So from tomorrow, you're gonnamanage them.
Francoise von Trapp (16:59):
There you go.
Subu Iyer (17:00):
Yeah, so so all of a sudden, right, I have this
300-plus uh group of packagingengineers, and now you know they
kind of have to do what I tellthem to do, right?
And so now it is a question ofgetting education across and
getting them to understand whatwe're trying to do and so on and
so forth.
And I think one of the things Iasked them when I started this
(17:23):
job was, hey, uh can you show meyour packaging roadmap?
So what they said was very,very enlightening, and I think
this is a very important thingto remember.
And they said, Subu, we don'twork with roadmaps.
Okay, because when you're insilicon, you know the roadmap is
Moore's law, right?
Every year we know what to do.
(17:44):
So, but they said we don't dothis.
The chip guys come to us with aproblem and we solve it.
And so I said, and the nextnext guy, the next chip comes,
we solve that too.
And is that the same solution?
No.
So I you know, and so I said,look, the main thing we have to
do here is we have to have astructured roadmap so that we
(18:04):
can tell the chip guys what isgonna come about, okay?
And they can plan for it, andyou can plan for it, and this
makes everything smoother.
So I think you know that isstill not happened in the even
with the heterogeneous.
Yeah, even with theheterogeneous T you know, TSMC
does it one way, these uh theguys do it this way, and you'll
(18:25):
see that like the OSATs, forexample, are trying to get into
the business, but everybody hasa different um different
roadmap.
Yeah, okay, and these roadmapsdon't talk to each other, they
don't flow from one to theother, and so on and so forth.
Now, and the reason why TSMC iskind of successful is their
roadmap is self-consistentwithin their ecosystem.
Francoise von Trapp (18:47):
Right.
Subu Iyer (18:47):
And that is that works for sure, as long as
you're willing to go theretoday.
Francoise von Trapp (18:52):
But it's also a problem because you can't
have one company dominatedominate because if something
happens to that company, we'rescrewed.
Subu Iyer (19:00):
Yeah, and it's also it's also not good for
innovation, right?
I mean, yeah.
So I think roadmapping is good,and that's one of the reasons,
right?
We started this uh uh effortwhich is funded by the
government actually calledMRHIEP, the manufacturing
roadmap for heterogeneousintegration and electronics
packaging.
And that roadmap is actually onour website.
Anybody can go and look at it.
(19:20):
And we uh when when I was inthe NAPMP, that was the roadmap
we followed.
Francoise von Trapp (19:25):
Okay.
Subu Iyer (19:26):
Okay, and uh that is uh the roadmap I think the
industry should follow.
Francoise von Trapp (19:29):
Did you ever see Back to the Future?
Yeah, long ago.
Yeah.
Do you remember Doc says wherewe're going, we don't need
roads.
Yes.
Okay, so since you brought upthe NAPMP.
Subu Iyer (19:42):
Oh, okay, I and we don't have too much time left.
Francoise von Trapp (19:45):
I did want to get there, and you brought us
there, so I'm happy about that.
You said today you showed aslide that talked about the
NAPMP structure and the sixhardware eco and ecosystem
thrusts and piloting facilityand prototyping challenges, and
then you had all these questionmarks across it because all of
this because of the currentadministration has scrapped the
NAPMP, but you told people don'ttoss this work, it's still
(20:09):
important.
Can you talk about that alittle bit?
Subu Iyer (20:11):
Correct.
First of all, let's be clear.
I don't think the governmenthas tossed NAP and P.
They are redefining it andprobably rebranding it.
And every new administrationwants to do that, right?
I mean, uh they want to kind ofmake sure that it reflects
their sort of ideas of what isuh what is important and so on.
And and that's perfectly okay.
(20:32):
I don't have any issue withthat.
I I think the new uh BAA that Ihave seen is broad enough and
includes uh in a very broadsense, you know, heterogeneous
integration, packaging, assemblytests, and all this stuff, you
know, which are which are thewhat these these five areas used
to address.
So I think, you know, myfeeling, okay, and I mean time
(20:55):
only will tell, is that theideas of the original thing are
still sort of valid and fitnicely actually into the new
structure.
And if you look at wherethey're pushing, right, the the
the new BAA is pushing AI.
Okay, it's pushing a wholebunch of other things.
But the AI push, okay, isclearly something that will be
(21:20):
driven by advanced packaging.
And you're not gonna get thoseresults, okay, if you want to
reduce the power and all thatgood stuff.
You're not gonna get thatwithout, you know, uh basically
following the kind of structureand the ideas that I presented
that I developed when I was inthe NAP and P and also what I
(21:41):
sort of briefly presented heretoday.
You have to address thesubstrate issue, you have to
address the equipment newequipment that is needed for
this, you have to developprocesses for it, you have to
address thermal and powerdelivery, you have to address
connectors, and you have toaddress the chip right problem,
and you have to address the EDAproblem.
(22:02):
Okay, those key elements arekey are still very important.
And you also need, I mean, it'snot enough to actually just
build these in abstract.
Okay, you need a place to buildthese.
Okay, and if you don't have aplace to build this, you'll have
to go to China, you'll have togo to Taiwan, you'll have to go
somewhere else to do it.
And so it is really, really,really, really important that we
(22:25):
provide a safe place where uhinnovators in this country, and
we have the best innovators inthis country, can actually sort
of try out their ideas withadvanced packaging and be able
to get parts back, test them,make sure they work, and that
will lead to bigger and biggervolumes and manufacturing in the
(22:46):
US.
So I firmly believe, okay, thatthe strategy that we put
together with my colleagues whenI was in the NAPMP, and what I
kind of re-enunciated today,okay, are vital for making the
country competitive, okay, in inadvanced packaging.
(23:09):
And one thing is very clearnobody, no country, no uh
country has a monopoly oninnovation.
Right.
Okay.
And today, right, I would sayhalf my students come from
outside this country, right?
And uh it's more and moredifficult nowadays with all
these restrictions for these forus to accept students from
(23:30):
certain countries.
And this is going to be a veryserious problem.
Big, big one the way we've beeninnovative is we've been able
to attract the best talent.
Francoise von Trapp (23:39):
Are the H1B visas impacting students?
Not really.
Subu Iyer (23:43):
They don't they don't impact students, okay, but they
do impact the ability of thesestudents to get jobs.
And if they don't get jobs andthey don't get these visas,
okay, they're going to go backto their home country.
So here we are, we put a lot ofeffort educating them, they
become fantastic engineers, andwe say go back to your country.
Right?
And um and there are jobs inthose countries, okay?
Francoise von Trapp (24:07):
And isn't it getting to the point where
there's other places they canget their education besides the
US as well?
Subu Iyer (24:12):
That is also a big, big danger.
One of the big big uh sort ofexports we have is actually
higher education.
Okay.
We have easily one of the besthigher education uh
infrastructures in the world.
And uh if we kind of dismantleit, right, and we are
dismantling it in many, manyways, I'm not saying that the
universities are not to blame.
(24:33):
The universities have actuallybrought a lot of these problems
on themselves by sort of notfocusing on the real problems.
So, for example, right, and andpart of that is actually the
engineering faculty who have notparticipated, and the science
faculty and the medical facultywho have not participated in the
(24:56):
administration of theuniversity.
So if you look at theadministration of the
university, it is dominated bythe humanities, the
philosophers, the sociologists,and so on and so forth.
And they have a differentagenda, okay, which is a very
idealized agenda, which is verydifficult to actually implement,
right?
Like if you think about likefree buses and free food and
free groceries for all.
(25:17):
I mean, that's not happening.
You cannot sustain that.
Right.
But the people who have to sortof uh put uh the voice of
reason here are notparticipating.
And so the universities havesort of swung that pendulum to
the other side, but the reactionis also not correct because by
withholding funding touniversities, those people don't
get government funding.
(25:38):
Okay, they don't have researchprojects, their overhead is very
low.
So they have plenty of time totalk about these problems.
The guys who get affected arepeople like me, okay, who
basically say cannot fundresearch.
It costs me $100,000 to fund aPhD student per year.
Wow.
Yeah.
Yeah.
So and you know, and for anexperimental number uh PhD,
(26:01):
right, which is what we need toget people skilled in
manufacturing, that number isalmost twice.
So I computed that it takesabout a million bucks to get a
PhD out of the door.
Wow.
No, we spend a million bucksand we say go back.
We're not gonna give you a job.
This is complete nonsense.
Okay, and unfortunately, right,the people who are in power
(26:23):
here do not seem to appreciatethat.
You cannot get a guy fromWalmart to be like a top-notch
Sundar Pichai at Google.
All right?
Francoise von Trapp (26:33):
Yes.
Subu Iyer (26:33):
I mean, and no arguments from me.
Yeah, and this is the problem.
Yeah, we're we're educatingthese people, we need to
benefit.
Right.
Okay.
I'll stop.
Francoise von Trapp (26:43):
No, I mean, I would talk to you all day
about this, but I can't.
So um I am just so happy tohave had this opportunity to
talk to you one more time.
I've really enjoyed ourconversations over the years, so
thank you for joining me today.
Subu Iyer (26:56):
Yeah, and it's been a pleasure talking with you and
uh having this free free-rollingconversation.
Okay.
Francoise von Trapp (27:02):
And then people get to listen to it.
So that's the fun.
I mean part.
All right.
Okay, thank you so much.
Thanks, Subu.
Subu Iyer (27:09):
Bye.
Francoise von Trapp (27:13):
For several years at the IMAP Symposium,
we've been following theevolution of co-packaged optics
as they make their way tocommercialization.
Now, this year, Jan Varnemanfocused her Wednesday evening
panel on the topic.
The topic was building theecosystem for co-packaged
optics.
And joining me to talk aboutthis are three of the panelists,
and they're all from our membercompanies.
(27:34):
I have Mike Kelly of Amcor, TolLiteken of Fraunhofer IZM, and
John Knickerbacher of IBM.
Welcome to the podcast, guys.
John Knickerbocker (27:43):
Thank you.
Thank you.
Francoise von Trapp (27:45):
Before we dive into the topic of the day,
can you each just tell us alittle bit about what you do at
your companies and how you'vebecome involved in co-packaged
optics, why you're interested init?
Mike Kelly (27:55):
Uh hi, my name is Mike Kelly.
The question is how do I gotinvolved in Silicon Photonics?
So I think like all of theprojects that we undertake, it's
something that our customersare interested in, and we need
to get prepared.
So uh Silicon Photonics, aseveryone knows, has been around
for a long, long time.
We've worked on it on and offfor you know years, I would say
(28:20):
around a decade probably.
What's completely aggravatedthe situation is all this
bandwidth requirements in thedata center for AI, and we're
looking to help our customershandle that in uh in a better
way than what what is possibletoday, staying completely
electrical.
So I think that's kind of thebackground on it, and uh, you
(28:40):
know, we're looking forward tomaking progress.
Francoise von Trapp (28:44):
Okay.
Tolga?
Togla Tekin (28:46):
Hi, my name is Tolga Tolga Tekken.
I'm in charge of the photonicplasmonic systems group in front
of RyCM.
Background optical signalprocessing.
I get introduced to photonicsin back to 95 since then working
on photonics, plus the onpackaging side.
So what we observe is actuallythe main bottleneck, off-chip,
(29:11):
off-core interconnects havinghigh bandwidth, low latency, and
high density interconnects.
So the promise is thatphotonics helping out with the
entire development coming fromtelecom, right?
John Knickerbocker (29:25):
That's right to leverage that.
Francoise von Trapp (29:27):
Okay, and John?
John Knickerbocker (29:29):
So uh maybe I'm a little different.
Uh my name is JohnNeckerbacher, and uh um I'm with
IBM Corporation, and I've beenuh developing advanced packaging
technologies for over fourdecades.
And uh and so uh a part ofthat, uh on and off through the
career has been uh various um uhtest vehicles that that uh
(29:51):
drove uh photonics and and uhoptical interconnect.
And over the last uh severalyears, uh I've I've led a team
that has uh Been driving some ofour um next generation
technologies, as we call it, uh,in terms of CPO technology and
advanced photonics.
And so, you know, I don'tactually consider myself a
photonics expert.
I'm not but I'm a packagingexpert, and we have many uh
(30:15):
photonics experts uh uh on ourteam.
And and so, you know, thecomplexity of photonics and and
co-packaged optics is such thatyou really need to have a broad
range of experts.
This technology is uh hascaught fire very recently, but
we started this more than adecade ago, back in the 2009
(30:36):
time period, and uh thesetechnologies kind of evolve from
a laboratory level, and uhultimately we want to go from
laboratory to prototyping andmanufacturing to help ourselves
and clients meet their needs.
And so that's how I've beeninvolved, uh helping to drive
this over the last severalyears.
Francoise von Trapp (30:55):
You know, as long as I've been in this
industry, which is 15 to 20years now, um when I started
hearing about photonics, fromwhat I understand, the issue has
always been on the packagingside.
You could do chip to chip.
The challenge was getting thesignal off the package.
You know, the conversation overthe years has ebbed and flowed
as developments happened andalso you know, drivers came
(31:19):
along, and it seems like nowwe've got this, as it always is
in the advanced packaging sideof things or in the industry
when there's a need for it, thatsuddenly there's this burst of
activity because now there's noother way to do it, or it's
going to solve a big industrychallenge.
So, just for the purposes ofour audience who may not
understand what co-packagedoptics are, we've got the
(31:40):
silicon photonics.
Um, what do we mean byco-packaged optics?
John Knickerbocker (31:46):
I've heard many different versions of what
people think co-packaged opticsis, and I I um uh looked and
found an ANSIS blog from uh2024, and I thought uh it very
well represented what uhco-packaged optics is, at least
from my vantage point.
This blog basically calls outthat CPO is uh an approach to
(32:09):
address the growing challengesaround bandwidth density,
communication latency, copperreach, and power efficiency uh
to support uh the nextgeneration optics and
electronics requirements.
To me, that really addressesthe key points for CPO.
Francoise von Trapp (32:26):
So basically, we're replacing the
copper with fiber optics in someway.
John Knickerbocker (32:32):
The reach of copper is challenging to go
beyond the one to two literrange, and optics is has that uh
opportunity to go that longerdistance and start to bring in
very high bandwidth and asenergy comes down to support
that in a power efficient way.
Mike Kelly (32:51):
I like your definition a lot.
I think that you know, whencoming at it from a strictly
packaging standpoint, when we'retalking about co-packaged
optics, it really gets down toyou know the provisions that
need to be inside the packagesomewhere to interface to the
the optical path.
And that's taken many formsover the last decade, but but it
(33:16):
seems like we've consolidatedon something now that makes
sense uh and is extensible.
And so now I think the the keyreally is making sure that we
have a a path to higher volumeso that it can be more or less
mainstream.
Seems like the the vision forthat is now nearer in than it
(33:36):
was ten years ago for sure.
So that's kind of what it meansto me.
Francoise von Trapp (33:40):
Okay.
Togla Tekin (33:40):
Indeed, the losses in ref transmission line
increase with the distance,right?
And for the time being, theelectro optical conversion takes
place at the edge of the panel.
So that's in the range oftwelve inches.
Right?
Would somehow corresponding to10, 12, 12 dB loss over there?
(34:03):
Around 60 gigabolt per second.
That that that's the metric,right?
So if it can come closer withelectro-optical conversion,
closer to the ASIC, whichrequires a higher bandwidth, so
then we'll that transmissionline will not consume so much
(34:24):
power.
Right?
It will be saving on that end.
That's that probably will yieldfour times minimum power
saving.
Francoise von Trapp (34:36):
So the driver for going to co-packaged
optics is primarily power?
Togla Tekin (34:42):
Basically the limitation of RF micro
transmission lanes.
Okay.
Francoise von Trapp (34:47):
There's a difference, right, between the
silicon photonics and theco-packaged optics.
Mike Kelly (34:52):
Power is definitely a part of it, but what the
industry needs is a lot morebandwidth for all these
disaggregated systems inparticular that are AI-centric.
Francoise von Trapp (35:02):
Right, okay.
Mike Kelly (35:03):
You've got to have more bandwidth for the next
generations, and this is the thebest way to achieve that.
Okay.
Togla Tekin (35:10):
I mean, let's refer to data center switch ASICs, 50
tera, 100 tera switches, right?
So the Redix on is limited.
It is SARDAS.
So that means for the timing,SARS speed is following the
bandwidth requirement of datacenter switches, right?
(35:31):
But each increase of SARDISspeed causes additional power.
So and then bringing all thosetogether requires a closer
integration of a OA conversioncloser to the switch ASIC
(35:53):
minimum.
Francoise von Trapp (35:55):
What's missing from the infrastructure
today that would make itpossible or that that's keeping
co-packaged optics from reachingcommercialization?
John Knickerbocker (36:07):
So I I think there's many obstacles to
really climb that curve and gofrom small volume research
demonstrators to volume that isused broadly across many
applications.
So I think one of the keydrivers now is AI, right?
(36:30):
And AI training and AIinference are very important.
And the bandwidth that's neededto support that effective
compute and especially theenergy being consumed to do even
just one training um algorithmcan take months at the present
(36:50):
time and significant energy tobring out these billion
parameter models algorithms thatsupport that information that
people want to access.
And the problem with that isthat's just the first turn on a
training module, and yet youoften need to go through more
(37:13):
iterations of fine tuning.
Uh likewise for inference uhoperation.
You also want to go throughsome iterations of fine tuning
and be efficient in this.
So as was stated earlier, I Ithink the value of CPO is both
in the bandwidth it can bring tothese applications that can
reduce the amount of time andenergy required for both
(37:35):
training and inference andfine-tuning, as well as the
ability to support theinterconnectivity of these uh
disaggregated systems across thedata center at longer distances
than can be done today withmany, many more GPUs and modules
that need to be connected inorder to be efficient.
Francoise von Trapp (37:54):
So, as far as a data center application,
for instance, a large languagemodel.
John Knickerbocker (37:59):
It's more about being um efficient with
energy and uh driving these verylarge language model algorithms
are just power hungry and needlots and lots of GPUs from you
know from hundreds to thousandsto tens of thousands.
And when you get at that scale,the uh the amount of energy
(38:20):
that's being consumed to createthese models and run them
through to be effective is justtoo large, right?
And so you can either reducethe algorithm size, right?
I think there's room for thatas well, or you can also uh
create this higher bandwidthconnectivity between these GPUs
so they can be uh more effectiveand get to the answer quick
(38:43):
more quickly.
And each is probably the rightway to go.
Uh some some combination.
Mike Kelly (38:48):
I think just a little fine-tune on that is like
you mentioned earlier, thealgorithms are so large.
Now they're trillions ofparameters, and so there's an
immensely greater amount of datamovement in the data center,
back and forth from thedifferent memory domains
surrounding all these GPUs.
So the amount of data thatneeds to go back and forth as
(39:11):
you train a model, or even whenyou infer from an existing
model, is orders of magnitudelarger now that LLMs have come
onto the scene.
This is still kind of attackingthe question why all the
interest in silicon photonicsnow?
Because the amount of data thatneeds to get moved around
between different GPUs andespecially between different
(39:34):
memory blocks in that GPU spaceis becoming and will become
larger than you can reallyhandle in copper, than you can
handle electrically.
It just we won't be able togrow the capability of data
centers if we stay electricallyconnected.
It's going to need to movebeyond that to get the amount of
(39:54):
bandwidth again to be able totackle these larger models.
Francoise von Trapp (39:59):
Okay, so we're understanding why the why
and what the driver is.
So, what's missing from theinfrastructure preventing us
from being able to adopt this?
Mike Kelly (40:09):
The thing that's been missing for a long time is
a really cost economic way tobuild silicon photonics
transceivers so that you can usethem ubiquitously everywhere.
That's work in progress still,it's not a done deal.
But a lot of the big umhardware companies that are
working on making this possiblehave put the stake in the
(40:31):
ground.
I have to have this.
And so, you know, I think it'senergized the the industrial
community.
Come around and say, we gottamake this happen now.
It's it's urgent, it's notoptional.
And to me, that's thedifference today than five years
ago or ten years ago.
Togla Tekin (40:48):
But so I would like to somehow differentiate,
right, the traditional datacenter are hard hyperscalers.
Francoise von Trapp (40:56):
Uh-huh.
Togla Tekin (40:57):
The CPO concept started over there.
With AI data center, sure, weneed high bandwidth inside the
data center, but additionallythe lowest latency in order to
have computing run parallel.
So I can probably give somenumbers on traditional data
centers, hyperscalers requiringthe acceptance of
(41:23):
electro-optical transceiversfrom globals is expected one
dollar per gigabit per second.
So if you can meet that target,you are in.
Otherwise you cannot be in thegame.
So possibly it will be similarin AI data centers, I could
(41:46):
imagine.
But since the requirements aredifferent, that several GPUs or
XPUs to be connected at the sametime.
It's more beyond than beyondthan the co-packaged optics.
So it's possibly in chipletlevel to be realized.
I mean, that's slightlydifferent.
(42:06):
I I will see more advancedthinker.
Francoise von Trapp (42:10):
Okay.
So we're gonna see firstimplementation of co-packaged
optics then in AI data centers.
Togla Tekin (42:18):
In traditional ones.
Francoise von Trapp (42:21):
You'll see it in hyperscalers first.
And when do we think when do wethink that's gonna happen?
Togla Tekin (42:29):
I mean if you meet one dollar per gigabit per
second today.
For the time being uhpluggables 400g is the right
current market.
Tomorrow 800 G.
So the vendors will be probablykeeping delivering those.
Mike Kelly (42:51):
Yeah, the pluggables you're talking about.
Yeah, you're right.
If you include pluggables intothis discussion, that's already
high volume, right?
So but when you're talkingabout co-packaged on in a large
GPU processor or CPU, that'sthat's different.
And that is still is still lowvolume.
Francoise von Trapp (43:17):
Okay.
So is it safe to say that wewill be having similar
conversations about this in fiveyears?
John Knickerbocker (43:26):
Or in five years so I think the the number
of optical connections permodule and and the bandwidth is
gonna climb over that timeperiod.
Yeah.
But I think uh it's gonna takeum, you know, pull from the
industry uh to you know climbthat curve.
(43:48):
So as any new technology goesfrom smaller volumes and
specialty, you start to get morestandards created and more
competition and more volume asthe uh the demand goes up and
and that drives itself in abusiness cycle to, you know,
(44:09):
winner is the one that givesthem a a cost-effective
solution.
And and so that's part of youknow, not just the technology,
but the business cycle tosupport a system requirements
that's cost effective and can bemet in the volumes of supply
chain that are needed.
And so all those things, likewhat's the exact timing?
(44:30):
Good question.
Um but I think over the nextfive years, if that's the time
horizon that that uh is beingdiscussed, that uh the the this
technology is gonna advancesubstantially in that time
period and beyond.
And so so again, I think you'regonna be talking um much higher
bandwidth, uh uh you knowpower, I think, um will come
(44:51):
down, but it may be secondary tostart with and then later
become a more important driver.
Uh so so again, I think allthese things are gonna advance,
and so it's actually veryexciting times uh because I see
the you know the the this topicarea is just caught fire in the
industry, and and so you've gotmany people working on it, and I
think it's gonna uh you know goat a faster pace.
Francoise von Trapp (45:13):
Well, then I think it's gonna be great to
continue having thisconversation and and following
this, I'm sure it'll be thetopic of many future IMAPs.
Well, thank you, all of you,for joining me today.
John Knickerbocker (45:26):
Thank you.
Thank you.
Francoise von Trapp (45:31):
Next time on the 3D Insights podcast,
we've got more from IMAPSymposium 2025.
You'll hear from one newcomerto the industry about why he is
interested in the field, andalso a group of students talk
about their experience at thisyear's IMAP.
There's lots more to come, sotune in next time to the 3D
Insights podcast.
The 3D Insights Podcast is aproduction of 3D Insights LLC.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
CrimeLess: Hillbilly Heist
It’s 1996 in rural North Carolina, and an oddball crew makes history when they pull off America’s third largest cash heist. But it’s all downhill from there. Join host Johnny Knoxville as he unspools a wild and woolly tale about a group of regular ‘ol folks who risked it all for a chance at a better life. CrimeLess: Hillbilly Heist answers the question: what would you do with 17.3 million dollars? The answer includes diamond rings, mansions, velvet Elvis paintings, plus a run for the border, murder-for-hire-plots, and FBI busts.