April 16, 2025 • 35 mins
Dr. Faisal Mahmood, Associate Professor of Pathology at Brigham and Women’s Hospital and Harvard Medical School, joins hosts Raj Manrai and Andy Beam on NEJM AI Grand Rounds to explore the frontier of computational pathology. From pioneering foundational models for whole slide imaging to commercializing a multimodal generative AI copilot for pathology, Faisal shares how his team is redefining what’s possible in digital diagnostics. He discusses the power of open-source culture in accelerating innovation, his lab’s FDA breakthrough designation, and how generative AI could trigger widespread digitization in pathology. Faisal also reflects on his creative approach to problem selection and offers a vision for a future shaped by patient-level foundation models and agent-led computational biology.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:04):
I think that making that entire code basepublicly available, and it's been a common
theme in all of our work. We always makethe code, everything as much as we can, and
as much as is allowed, publicly availablehas been a big accelerator for us. And
it has led to all kinds of improvements.
Things that I had neverimagined because once we made
CLAM publicly available, you know,everyone from the entire community
(00:26):
started contributing, and giving usfeedback as to what we can improve.
And of course, we do a lot of improvementinternally as students learn what is
the best way to do what over time.
But we also had this community feedbackthat led to a lot of systematization,
computational improvement, just ideasfor how we can improve the entire
(00:46):
pipeline and, and, and scale it.
It was sort of an internal battle to try to make some of
the larger foundation models thatwe had built publicly available.
We had the innovations office andcorresponding legal people involved
in trying to understand what isit that we are trying to make
public. And once they understoodthat the data is not being made
(01:09):
public, no identifiable labelswere used in the training.
Welcome to another episodeof NEJM AI Grand Rounds.
I'm Raj Manrai, and today weare excited to bring you our
conversation with Dr. Faisal Mahmood.
Faisal is an Associate Professorof Pathology at Brigham and Women's
Hospital and Harvard MedicalSchool, and a leading computational
(01:33):
researcher working on applying AI tomedicine and in particular pathology.
We had a chance in the episode todig into Faisal's work on creating
new pathology foundation models,
his experience starting acompany while based in academia,
and his approach to selectinginteresting scientific problems.
Faisal's lab has been really,really productive and he shared
some of their secrets for success.
(01:54):
All in all, this is a reallyfun and insightful conversation.
The NEJM AI Grand Rounds podcastis brought to you by Microsoft,
Vis.ai, Lyric, and Elevance Health.
We thank them for their support.
(02:15):
And with that, we bring you ourconversation with Dr. Faisal Mahmood.
Faisal,
welcome to AI Grand Rounds.
We're excited to have you here today.
Oh, thank you so much for having me.
Faisal,
great to see you and greatto have you on the podcast.
So, this is a question thatwe always get started with.
Could you please tell usabout the training procedure
for your own neural network?
(02:36):
How did you get interested in artificialintelligence? And what data and experiences
led you to where you are today?
Yeah, my background is in electricalengineering and computer science,
and some of the earlier work I didwas around imagery construction and
electron microscopy and how to usecryo-electron microscopy images and
reconstruct them back in three dimensions.
(02:57):
And I did a lot of theoreticalwork on optimization to try to
make it happen. Deep learning
before it was called deep learning.
And then I did a postdoc at Johns Hopkins. And it was very different, more related to
medical imaging than biomedical imaging.
And I predominantly focused on endoscopyand synthetic data generation and how can
(03:18):
you use synthetic data generation to solveand answer some questions in endoscopy.
And then from there, I became interested inpathology and moved here at the Brigham,
where I've been for about six years now.
And the reason I really got interestedin pathology is because it was often
regarded as one of the last modalitiesin medicine to be digitized since there's
a lot of untapped potential there.
(03:40):
When I was at Hopkins, they were justgetting started with digitizing slides and
did some early work there and that'swhat I've focused on since moving here.
You know, something that struck me is thatI think a lot of our guests have often
sort of early stories or early exposuresto artificial intelligence that, whether
through science fiction, through readingor through movies that they then now get
(04:00):
to revisit and engage in some of thesekind of wild ideas from childhood
and early on with their professional work.
And I'm just wondering if you couldtake us back like even further, right?
Even before that, a little bit.
Do you have any memories of earlyexposure to AI or interests that
are related to AI even beforeyou started your research career?
(04:22):
Nothing comes to mind off the, offthe bat, but yeah, I'll say this,
that I was always interested indoing a lot of data driven modeling.
It just made sense to me thatusing lots of data to answer
questions that are important would bethe way you make machines intelligent.
For a while in my field, in medicalimaging, there was a lot of work being
(04:43):
done on using handcrafted featuresand using all kinds of other modeling
techniques, but it was not scalable.
And just using lots of data, abstract feature representation,
that all sort of made sense to me.
Got it.
Great.
So, we want to actually dig into someof your work in AI for pathology.
(05:05):
And I honestly don't know where to start.
This is a huge compliment to you.
I think you've been justamazingly productive.
When I think of someone who's reallybeen at the forefront of AI for pathology,
you know, you're the first personthat comes to mind. And you've just
done this amazing, amazing amountof work for the past few years.
(05:26):
And again, this is a hard choice aboutlike, which paper of yours to start
with, but maybe I can start with
one that, out of personal interest,caught my eye a few years ago and
it's gotten a lot of attention that Ithink has also inspired a lot of folks.
And this is your paper that, andthis is another question that we have
to ask you at some point, which ishow you come up with these names,
(05:46):
but this is your paper on CLAM.
The title of the paper is "Data efficient and weakly
supervised computationalpathology on whole-slide images."
So, I think that title'sactually pretty informative.
This was published in NatureBiomedical Engineering a few years
ago, back in 2021, I believe.
Maybe we could start with that paper.
Could you tell us about like, first ofall, tell us what that title means, what
(06:06):
you were trying to do there, and maybealso take us back and give us a little
bit of the backstory on the paper itself.
Yeah, so, when I got started in thisfield, making some of these methods
data efficient was a big problem.
Lemme just give a littlebit of a background here.
So, glass slides, pathology glass slides.
When they're digitized, theyturn into gigapixel images.
(06:27):
So, these images are very, very largeand a single patient case can have
anywhere from a single slide to up to,you know, a couple of hundred slides
depending on what kind of case it is.
And the only label that you havecorresponding this entire case, a
multitude of very, very large gigapixelimages, they're almost like satellite
images, is the pathology report.
So, the relationship between your data andyour label is incredibly weak, and when
(06:52):
you have lots of images and single labels,
what comes to mind from a machinelearning point of view is that yes,
well, we can use multiple instance learning. But you know,
multiple instance learning can be inherently very data inefficient in
this case, because a very small regionfrom the gigapixel image can update
the weights and biases of the model.
And there were some work done beforeour 2021 Nature BME article showing
(07:15):
that you needed about 10,000 wholeslide images corresponding a single
class to train a model that would beequivalent to pixel level annotations.
So, data efficiency in this area was ahuge problem. And what my group inherently
does is that we develop solutions andmethods, computational methods that would
solve very targeted clinical problems.
(07:37):
So, this is a study that we didmore on the methods side, or
we try to solve this problem.
It's just being really focusedon how can you make this form
of supervised learning
more data efficient? And we useda number of different bells and
whistles at the time that were usedby conventional machine learning, like
attention, and models that were trainedon ImageNet, and just extracting features
(07:59):
from those models to make the setupinherently more, more data efficient.
But I think the really catalyzingfactor and why this article is so
popular is that we made all thecode and models and everything
corresponding to this publicly available.
So, people have really gone to townusing this tool for every major
organ system, every disease model,all the way up to forensics, to
(08:21):
wherever you have pathology images,
it's, it's been applied.
And—. Were you surprised by any of the uses of the model?
Yeah, we've been surprised thatnumber of times. People have used
it for plant pathology, things thatI just would've never imagined.
And at the time, the backstory isthat the real reason we did this
(08:41):
study, because I was very interestedin cancers of unknown primary.
So, when cancer metastasizes,it often becomes unclear
where it may have originated.
And identifying the primary originof tumor is really important because
most drugs are primary tumor specific.And these patients can't go in a clinical
trial because most trials would requirethat the primary be already identified.
(09:03):
So, we, we wanted to solve that problemand it's like a highly imbalanced,
multi-class multitask kind of a problem.
But there was no good way to solve itusing these very large gigapixel images
where the number of samples per patientcan vary and the number of cases we
have per patient is heavily imbalanced.
So, that's why we developed CLAM.And then we applied that to this
(09:27):
specific problem in cancer ofunknown primary, and that's a,
that's also a 2021 Nature article.
And we call it TOAD.
This is TOAD, right?
TOAD, yeah.
Yeah.
Okay.
So, how do you, I gotta ask now,how do you come up with the names?
So, it, it became just a theme.
So, so, I came up withCLAM 'cause I like— Okay.
—eating them.
Right.
So, okay.
So, so, so, so, it's, it'sone of my favorite foods.
(09:50):
So, I just came up with CLAM.
And it also made sense because it's,clustering constrained attention,
multiple instance learning. Nice.
But that sort of inspired my studentsto start coming up with names.
In the beginning it was all aquatic themes.
And one of my Ph.D. students who was from the big program, Richard,
(10:11):
he eventually created an internal packagethat we were using for quite some time.
It was called Fishing Rod.
So, you use the Fishing Rod tofish all these aquatic species.
Amazing.
Amazing.
I think you said something thatI thought was really important.
So, I just wanna dig into a littlebit, which is that you think part
of the reason that CLAM took off,and I think this has actually been a
(10:32):
theme in a lot of your work, right?
But one of the reasons youattributed to CLAM taking off was
releasing the model. Making it opensource. Making it available, making
it easy for people to build off of.
And I think this has been a, you know,I just, I'm immediately drawn to the
comparison with, what David Ouyangdoes for AI, for cardiology. He was
a guest we had on a couple months ago.
(10:52):
And, you know, a coupleother folks who've really
emphasized the importance of it.
And I don't think we can say thisenough because I think it's just,
it really changes the trajectory of
the paper afterwards, if people canreally build off of it easily, and if
they can, use it to do their own work.
And so, I'll commend you on that, but Ialso think it's worth just maybe talking
(11:14):
about that a little bit more, right?
Like, I think there's a lot ofthings that are attached to it.
You have, you're naming it right?
You're sort of, this is the wrong word,but you're making it an entity, right?
Productizing, that'sthe wrong word for it.
But you're making it sort of a thing.
You're releasing resources that accompanythat paper, not just sort of ending
at the science or the inference orthe findings, which are fascinating.
(11:36):
They're interesting, but you're alsomaking it sort of a engineering project
that people can then build off of.
So, maybe you could talk about that,like what are the ingredients here?
I think you've done this a lot.
You know, what are, what'sreally important to emphasize for
someone who wants to emulate that?
And, and, and maybe
if you could comment on the extent towhich you get institutional pushback or
support on this, because my experiencehas been that, like, if I train a
(11:59):
model on hospital data, the hospitalgets very nervous about potentially
releasing the model, not even the data.
So, I don't, I, I, if you haveany, uh, experience with that,
I'd love to hear about that, too.
Oh yeah, absolutely.
So, with CLAM, it was mostly the codebase because, we had some hospital
test data, but the models were largelytrained on the TCGA at the time.
(12:21):
I think that making that entire code base
publicly available, and it's beena common theme in all of our work.
We always make the code, everythingas much as we can and as much as is
allowed publicly available has beena big accelerator for us. And it has
led to all kinds of improvements.
Things that I had never imagined becauseonce we made CLAM publicly available
(12:42):
everyone from the entire communitystarted contributing and giving
us feedback as to what we can improve.
And of course, we do a lot of improvementinternally as students learn what is
the best way to do what over time.
But we also had this community feedbackthat led to a lot of systematization,
computational improvement, just ideasfor how we can improve the entire
(13:03):
pipeline and, and, and scale it.
And to your question, Andy, so, absolutely.
It was sort of an internal battle to try to make some of the
larger foundation models thatwe had built publicly available.
We didn't face any resistance. It'sjust a matter of understanding.
So, we had the innovations office andcorresponding legal people involved and
(13:25):
trying to understand what are, what isit that we are trying to make public.
And once they understood that the datais not being made public. It's a model
that's trained in a self-supervisedmanner, no identifiable labels were
used in the, in the training.
And there, were by the time wewere making the first self-supervised
large scale model public, there wereother examples from a lot of other
(13:48):
institutions, including from Stanford,that had already been made public.
And then they did a lot of digging and trying to understand what kind
of license should be used for this.
But eventually we did get the permissionsto make the make the models public.
Awesome.
I'd like to now hop to another one ofyour papers and continue the shameless
(14:08):
flattery here that Raj started andsay that like, there's like two
constancies in my Twitter feed.
One is that Elon Musk has done orposted something crazy, and two
is that Faisal has posted anotherNature paper on AI for pathology.
So, so, thanks for being that like steadyrock in my Twitter timeline.
Faisal, so this paper I think is
interesting
(14:29):
not just because the work was firstrate, but also because of the model it
suggests for AI going forward in medicine.
And so, the paper is "A multimodalgenerative AI copilot for human pathology."
And so, this was your PathChat paper.
So, one, I'd love to like for youto tell the listeners what PathChat
is, what it does, and whether or notco-pilot is a mental model that we should
(14:52):
have for AI and medicine going forward.
Yeah, absolutely.
So, the story of this study is thatI often call it an accidental study.
So, we started by, so, we'resolving all these entrusting,
supervised problems, and they'rebecoming more and more complex.
We started by solving very small, supervised classification
(15:14):
problems. And then they became more complex for solving multi-class,
multitask problems still, verysupervised. And, but the backbone behind
it is that because these images arelarge, we pre-extract features using a
ResNet that's just trained on ImageNet.
And then everything that was happening inconventional computer vision and machine
learning in general with, with selfsupervised learning, it became clear.
That you can find just amazing applicationpathology where you have all these rare
(15:38):
diseases, clinical trials, situationswhere you don't have enough data
available for, for supervised learning.
And we started to train large selfsupervised models, maximizing for
diversity, trying to collect every known
human pathology indication that wouldexist in our archives. Because it was
established using a number of differentstudies, including DINOv2 from Meta
(16:00):
showing that the diversity of data mattersway more than the quantity of data.
So, we max, try to maximize for diversityand we published two foundation
models in Nature Medicine and
once we had that, it obviously accelerateda lot of research we're doing and
just building more supervised models.
But another thing that we could dowith those self-supervised models is
because it can extract rich featurerepresentations now from pathology
(16:23):
images, is that we could, if we haveenough instructions, we could train a
multi-model, large language model, right?
So,
the goal was that can we have asingle multimodal, large language model
that can cater to all of human pathology?
And there's a lot of philosophy behindit as to why we think that's possible.
Like, companies like OpenAI aretrying to build a single multimodal,
large language model that willcater to human knowledge and
(16:46):
pathology is much more specific.
And we, if we have enough data
there's a high chance that you canconverge to a model that would do well
around everything around pathologies.
Just that data is locked upand it's not easily available.
And the other challenge was thatnow we have to go in the opposite
direction that pathology reportsdon't have enough morphologic details.
(17:06):
And a perfect human pathology chatbotor a co-pilot should be able to answer
questions at any magnification level.
It should be able to answer questionsfor specific regions in an image, for a
single image, or a multitude of images,
to be really, really useful.
So, we had to collect a lot of datamanually, but we also made use of
data that was used for teachingpurposes at the Brigham and MGH.
(17:28):
So, we have a lot of colleaguesand friends who contributed data
to this, but it was also a verytargeted data collection effort,
including a lot of other institutions.
And then we eventually had avery large instruction set.
We trained the multimodal largelanguage model, and we eventually
had a chatbot that we could usefor any pathology indication.
And then the assessment of the model wasmore difficult than building it because
(17:51):
we already had the foundation model and wespent a lot of time collecting the data.
We built
the model, and this entire processtook about a year. But it took
another year to just come up with thatevaluation paradigm, that would be
rigorous enough, took a lot of inspirationfor how, for what was happening in the
large language model research, both formedicine and in other critical areas.
(18:11):
And yeah, and then it waspublished last year around June.
We have a lot of demos aroundit, publicly available.
So, maybe I just wannaconnect a couple dots there
based on what you said. So, like,in large language model development,
there's often this very expensive thingcalled pre-training where you create
the base model and that creates amodel that has seen a lot of data and
can do many things, but it doesn't do
(18:33):
many things well.
And specifically, one ofthe things that it doesn't do
well is interact with people.
Yeah.
And so, after this large foundationmodel has been created, there's this
set of techniques called post trainingthat's often applied to the base model.
Some of it is supervised fine tuningor instruction. Fine tuning, like
you mentioned, where you actuallyteach the model to follow instructions.
(18:53):
Sometimes you do reinforcementlearning from human feedback.
There's this whole kind of like dark art—
Yeah.
—to making these models useful.
And it sounds to me like what you weredoing was sort of a one-for-one analog of
going from base GPT to ChatGPT whereyou took this very powerful foundation
model, but made it usable for people.
Is that a fair characterization?
Oh yeah, absolutely.
(19:14):
So, I often say that we didtwo stages, but not three.
So, a typical three stage processwould be that you have the strong backbone
model, you do instruction tuning on topof it, and then you do reinforcement-
learning-based fine tuning whereyou get a lot of human user feedback.
The model can generate many responses andthe human would say, which one is better?
And then you use that to, tofurther—. I assume that Nature
paper will be coming out this June.
I'm sure that, that's somethingthat you're working on now.
(19:35):
We're working on it.
It's, it's tedious and difficult because—.One Nature paper a year or I'm
not impressed anymore, Faisal, youset a really high, a high standard.
Yeah.
And so, just to, okay, so, uh, I believethat recently PathChat has become
the object of commercialization fromyour lab, that there's a spin out
(19:56):
around that, and that you received anFDA breakthrough designation for it.
Can you tell us more about that?
Yeah, absolutely.
So, two of my Ph.D. students, Max Liu
from MITE and Richard Chen fromfrom the Harvard Big Program, they had
decided early on that they wanted tostart a company once they graduate.
And PathChat seemed like a goodopportunity because it became
(20:16):
clear once we sort of announced it when it was
a preprint, that there was justa lot of interest. And I guess
it was also around when generativeAI, I mean it still does,
a lot of hype around it.
And it is truly useful in the sense thatit could be great for training, but also
it has the capability to impact the entirehorizontal of the pathology workflow.
(20:37):
Right?
Because it, it's a chatbot,but you can also constrain the
outputs to do other things.
Like it can become a universal triagetool, or it can look at a case and
suggest what ancillary tests and IHCs toorder, and those can already be ordered
before a pathologist is looking at it,or it can write the pathology report.
So, so, all of these auxiliarythings it can do can have a
(20:57):
lot of impact in pathology.
So, Richard and Max startedthe company together with some
of my other colleagues, and thenwe were thinking that what's the
best regulatory pathway for this?
And what my thinking and thinking ofmy colleagues in general is, how does
digital pathology get universal adoption?
Because the digitization rate aroundthe country is just around 4%.
(21:22):
It's very limited, and there area number of reasons for this.
It's expensive to digitize these images.
It's expensive to store them, but itobviously makes sense to everyone because
the research benefits down the line are just enormous because we
haven't discussed the discovery aspectaround these, images, but we think that
if there is sort of a killerapp, a killer AI tool, that would
(21:42):
make everyone's life easy peoplewould go digital very quickly.
So, we think that a universal triagedevice for pathology could drive a
lot of digital pathology adoption.
And the reason is that pathologistswould often look at many cases.
And within each case there wouldbe lots of slides and only a
fraction of them might be positive.
And if the device can tell you whichones are negative, it can make life
(22:04):
easy for a lot of people.
And there's a good predicate to this.
A lot of people think that AI forpathology devices have recently begun
to be approved by the FDA, but therewas one that was approved in 1998.
It was called Auto Pap.
So, it was just a camera attached to amotor, taking images from a slide and
doing a line profile through the cells.
(22:24):
It was for pap smear detection,and that was around when
there were a lot of pap smears tobe, to be analyzed, and there was a
shortage of of experts to do so.
So, the FDA approved this.
And it has something calledan NSR or no second review.
So, this meant that all the negativeslides could be screened or triaged
(22:44):
out and the experts would only haveto look at about 10% of the negatives.
So, that's one of the ideas and PathChatcan inherently be a very good triage tool.
And it's also
an assistive tool.
And the reason we wanted to getbreakthrough device designation is because
this is a relatively new technology andbreakthrough device designation gives
(23:04):
you access to the FDA in the sense thatthey would meet with you regularly and
design like a pathway towards approval.
And it's often the best course ofaction for new technologies and devices.
And generative AI is relatively new.
I mean, there are all these like openquestions as to how do you get a device
approved through the FDA that can make adiagnosis corresponding every indication.
(23:27):
So, that was the reason to get thebreakthrough device designation.
The FDA requires evidence around theefficacy of the device and that it's
truly sort of a quote unquote breakthroughto give that kind of approval.
So, that's what we essentially did.
And it was granted earlier this year.
Super interesting.
First of all, shout out RichardChen, who was an intern with me.
The first intern I think I eversupervised when I was a postdoc.
(23:50):
So, it sounds like he's goneon to spectacular things.
Yeah.
Second of all, one of the things thatI've learned from being in this space
and certainly from this podcast, isthat technology development is hard.
Commercialization isoften much harder still.
And so, how do you think aboutbusiness models around this?
Is there gonna be a reimbursement model?
(24:11):
Is this something like, I guess like howdo you think about sort of market value
or what even the market is in this case?
Yeah, that's a great questionand I know Richard has been
thinking about it a lot.
And I definitely agree with you.
Richard is one of a kind.
He was our superstar graduatestudent and did really, really
well, in the lab and is sort of
addicted to making digital pathologyand computational pathology just be
(24:33):
absorbed in the clinic, and he reallywants to change the clinical paradigm.
So, it's a really big passion of his.
I think that there's some benefithere from the predicate we have in
cytology that has been reimbursedfor a very, very long time.
So, reimbursement is,
no doubt, a challenge for, it's achallenge in AI for health in general,
and it sort of trickles down into pathology, but it might not be that
(24:55):
big of a challenge for a triage device.
The bigger challenge that we see isjust digital pathology adoption, how
quickly it would happen, and how muchof a trigger would FDA approval for
a primary triage would be. So,besides Auto Pap, all other more recent FDA
approvals for AI for pathology toolshave all been for a secondary analysis.
(25:18):
So, pathologist first looks atit, and then the machine learning
would essentially confirm.
So, if we do get approval for thisdown the line, it would be for
primary triage at the slide level.
And then we hope that that would sort ofdrive more digital pathology adoption.
Cool.
Awesome.
Thanks.
Okay.
I think we're ready to moveto the lightning ground.
(25:46):
Are you ready?
Yeah.
Yeah.
Ready.
Andy, did you wanna give us the introand the rules of the lightning round?
Sure.
In the lightning round we'llask you a series of questions.
It's up to you to decide howserious or non-serious to take them.
Short answers are rewarded handsomely,and long answers are punished viciously.
So, those are the rulesof the lightning round.
(26:06):
So, Faisal do you accept?
Do you accept the task?
Absolutely.
So, this one is a little bit of a,I would say, pet peeve of mine.
In that I have irrationally strongopinions on it, but in your opinion, do
you think that AI should be explainable?
To a degree.
I love, uh, one of the FDApresentations showing something.
(26:28):
It said that AI gives hints, andI completely agree with that, but
I don't think we should obsessover interpretability too much.
The performance evaluation, theclinical trials, all of that is,
is much more stronger evidence.
There's, there's a lot of otherthings in health care and medicine
that we can't really explain.
I agree with that
100%.
Cool.
This is just that, that answerwas music to Andy's ears.
(26:50):
I know.
So, Faisal, nextlightning round question.
What's your favorite novel?
Oh, uh, Norwegian Wood by Haruki Murakami.
Wow.
Nice.
Deep cut.
If you weren't a professor atan academic medical center, what
(27:11):
would you be doing with your life?
Oh, so, if I was not a professor at anacademic medical center? Or more generally
doing health care, like medical AI.
So, if in the counterfactual world, Faisalwas doing something completely different,
what would counterfactual Faisal be doing?
Oh, I'll be writing.
Oh, okay.
Novels, like the onethat you just mentioned?
(27:31):
Yeah, yeah.
Something like that.
have you written before? Have youwritten fiction or nonfiction before?
No, but that's what Iwanted to do as a kid.
Oh, wow.
Yeah.
Faisal, will AI and medicinebe driven more by computer
scientists or by clinicians?
So, I think, it'll be a balance.
So, if I just look at what I'velearned over the past decade, some of
(27:53):
the best projects in my lab have comeabout when computational trainees and
clinical trainees partner together.
And I've been very lucky to havegreat graduate students from
amazing Harvard and MIT programs,but also just great residents
and fellows working in the group.
And some
were just amazing withPh.D.s in computer science.
(28:14):
And I've also gone throughentire clinical trainings.
When they work together,that's when the magic happens.
Yeah.
Cool.
Alright.
Last lightning round question.
It's one that we ask most of our guestsand we always get interesting answers.
If you could have dinner with oneperson, dead or alive, who would it be?
Ooh.
(28:34):
Wow.
Uh, that's a, that's a tough one.
Tesla.
Oh, Nikola Tesla.
Yeah.
Nikola Tesla.
I mean, my background isin electrical engineering.
Just really inspired by all theout of the box thinking, but
still everything that made sense.
I wish it was possible.
Yeah.
And unfortunately overshadowed byEdison in many regards, despite
(28:55):
being his equal or greater, so. Yeah.
Yeah.
Cool.
Awesome.
Well, Faisal, you, yousurvived the lightning round.
Congratulations.
Thank you.
Alright, so, we have a couplebig picture questions to
throw at you before we wrap up.
Again, I think we've touched onthis a little bit throughout the
episode, but do you have a, like a,a unique ability to select problems
(29:16):
that have the dual properties ofbeing interesting, but also not
stuff that everyone else is workingon, and therefore you're able to do
meaningful work without momentum chasing.
And so, what is your method forselecting interesting problems
in such a fast-moving field?
(29:37):
So, I pick problems for which there'sno good solution right now, right?
So, I think that of course we can usemachine learning to replicate what humans
do and maybe do a little bit better,reduce intra observer variability.
All of that is great,but I just feel that
(29:57):
we as machine learning for healthresearchers can make a bigger impact if
we target problems that for which there'ssort of no good solution right now.
Right?
So, so, it was reflected in my postdoc,I worked on trying to see if we can
use machine learning to predict depth from2D conventional and endoscopes, right?
So, so, we generate a lot of syntheticdata and try to solve that problem.
(30:21):
If we're able to do that,you can essentially convert
any conventional 2D endoscope intoa 2.5D or a 3D endoscope.
And the same is reflectedin my more recent works over
cancers of unknown primary.
There, there are a numberof different solutions, but
there's no good solution.
And these cases are just, it takes sucha long time to diagnose these cases.
(30:44):
And the other large study wedid, it was back in 2022 for endo
myocardial biopsy assessments.
There's just huge intraobserver variability.
It has major issues with the downstreamtreatment, so, just problems for
which there is no clear solution.
People are much more amenable toadapt those models into clinical
(31:06):
practice versus trying to replicateor just build assistive tools.
Awesome.
Faisal, this is our last questionfor you, and maybe I can get you
to give us your future vision forhow medical AI is gonna evolve.
And specifically, I think we'd beinterested in your taking.
I think we've heard a lot about howyou select interesting problems,
(31:28):
how you've designed medicalAI applications for pathology.
But maybe you can reflect on and projectfor specifically translation into the
clinic, how you see medical AI shaping,reshaping, changing, not really changing
medicine over the next five to 10 years.
(31:49):
And, I think it would also be veryinteresting for you to tell us about
what you think are the key bottlenecks.
Are they social?
Are they technical?
Are they regulatory?
What are the key bottlenecks tomoving AI from where it is right now
to a greater role in clinical care?
Yeah, I think that there are twolines of development that would
(32:11):
happen over the next five years or so.
But and it always happens soonerthan I think so, so, it might even
happen in the next few years.
One of those directions is thatI think that we'll soon have
patient level models, right?
So, we're seeing that all these self-supervised models in individual domains
are becoming better and better atrepresenting those particular modalities.
(32:32):
And we're seeing some earlier flavorof multimodal, so, quote unquote
foundation models, where contrastingwith another modality improves
representation for all modalities.
We'll see that done in a muchmore holistic way, where the
patient's entire medical record.
It could be featurized into a singlefeature vector, and it would
have a temporal component to it.
So, if a patient who's on immunotherapyhas a toxicity event over the weekend
(32:54):
their feature vector has now changed.
And, and then all the downstreamvariables would reflect that.
And that would do essentially two things,would really enable a lot of early
detection, early diagnosis, and it canbe happening on the fly all the time.
And I think that achieving thisis very much possible today.
it's only a matter oftime before this happens.
Once this happens, we will alsohave the ability to use entire
(33:17):
clinical grade health care systemscale data for discovery, right?
Because if you look throughout thehistory of medicine, how do people come
up with new diseases, new subtypes?Is that someone goes in, looks at lots
of data and says that, well, this
particular, morphologic phenotype onan image or all of this categorization
correlates with outcome, and this issubtype A and subtype B and disease A and
(33:38):
disease B, and we have the opportunityto do that automatically on its own,
almost continuously, as the ology ofsome of these diseases evolves over time.
So, it'll become like automatic
discovery engines.
That's one thing.
And then the other thing I think that willhappen, and it's also in the short term,
is that with all the agents and agentworkflows, computational biology as we
(34:02):
know, it would just fundamentally change.Because what computational biologists
do today, where you have existing dataor new data that you have collected,
and you're often using existing toolsand you're patching together some code
that you write and using more tools,and sometimes you're developing newer
tools, would just be done on its own.
And a lot of hypothesis generationand testing and assessment
(34:22):
can also happen on its own.
So, I think those are the two big things that are, that I
think are just obviously coming.
So, just patient level foundationmodels that will enable both clinical
change as well as lots of discovery and then agent workflows that would,
that would just accelerate all kindsof computational biology research.
(34:43):
Alright, well I think that'sa great note to end on.
Thank you so much forbeing on AI Grand Rounds.
Yeah.
Thank you so much for having me.
Thanks for coming, Faisal.
Thanks.
Thanks guys.
This copyrighted podcast from theMassachusetts Medical Society may
not be reproduced, distributed,or used for commercial purposes
without prior written permission ofthe Massachusetts Medical Society.
(35:07):
For information on reusing NEJM Grouppodcasts, please visit the permissions
and licensing page at the NEJM website.
I think that making that entire code basepublicly available, and it's been a common
theme in all of our work. We always makethe code, everything as much as we can, and
as much as is allowed, publicly availablehas been a big accelerator for us. And
it has led to all kinds of improvements.
Things that I had neverimagined because once we made
CLAM publicly available, you know,everyone from the entire community
(00:26):
started contributing, and giving usfeedback as to what we can improve.
And of course, we do a lot of improvementinternally as students learn what is
the best way to do what over time.
But we also had this community feedbackthat led to a lot of systematization,
computational improvement, just ideasfor how we can improve the entire
(00:46):
pipeline and, and, and scale it.
It was sort of an internal battle to try to make some of
the larger foundation models thatwe had built publicly available.
We had the innovations office andcorresponding legal people involved
in trying to understand what isit that we are trying to make
public. And once they understoodthat the data is not being made
(01:09):
public, no identifiable labelswere used in the training.
Welcome to another episodeof NEJM AI Grand Rounds.
I'm Raj Manrai, and today weare excited to bring you our
conversation with Dr. Faisal Mahmood.
Faisal is an Associate Professorof Pathology at Brigham and Women's
Hospital and Harvard MedicalSchool, and a leading computational
(01:33):
researcher working on applying AI tomedicine and in particular pathology.
We had a chance in the episode todig into Faisal's work on creating
new pathology foundation models,
his experience starting acompany while based in academia,
and his approach to selectinginteresting scientific problems.
Faisal's lab has been really,really productive and he shared
some of their secrets for success.
(01:54):
All in all, this is a reallyfun and insightful conversation.
The NEJM AI Grand Rounds podcastis brought to you by Microsoft,
Vis.ai, Lyric, and Elevance Health.
We thank them for their support.
(02:15):
And with that, we bring you ourconversation with Dr. Faisal Mahmood.
Faisal,
welcome to AI Grand Rounds.
We're excited to have you here today.
Oh, thank you so much for having me.
Faisal,
great to see you and greatto have you on the podcast.
So, this is a question thatwe always get started with.
Could you please tell usabout the training procedure
for your own neural network?
(02:36):
How did you get interested in artificialintelligence? And what data and experiences
led you to where you are today?
Yeah, my background is in electricalengineering and computer science,
and some of the earlier work I didwas around imagery construction and
electron microscopy and how to usecryo-electron microscopy images and
reconstruct them back in three dimensions.
(02:57):
And I did a lot of theoreticalwork on optimization to try to
make it happen. Deep learning
before it was called deep learning.
And then I did a postdoc at Johns Hopkins. And it was very different, more related to
medical imaging than biomedical imaging.
And I predominantly focused on endoscopyand synthetic data generation and how can
(03:18):
you use synthetic data generation to solveand answer some questions in endoscopy.
And then from there, I became interested inpathology and moved here at the Brigham,
where I've been for about six years now.
And the reason I really got interestedin pathology is because it was often
regarded as one of the last modalitiesin medicine to be digitized since there's
a lot of untapped potential there.
(03:40):
When I was at Hopkins, they were justgetting started with digitizing slides and
did some early work there and that'swhat I've focused on since moving here.
You know, something that struck me is thatI think a lot of our guests have often
sort of early stories or early exposuresto artificial intelligence that, whether
through science fiction, through readingor through movies that they then now get
(04:00):
to revisit and engage in some of thesekind of wild ideas from childhood
and early on with their professional work.
And I'm just wondering if you couldtake us back like even further, right?
Even before that, a little bit.
Do you have any memories of earlyexposure to AI or interests that
are related to AI even beforeyou started your research career?
(04:22):
Nothing comes to mind off the, offthe bat, but yeah, I'll say this,
that I was always interested indoing a lot of data driven modeling.
It just made sense to me thatusing lots of data to answer
questions that are important would bethe way you make machines intelligent.
For a while in my field, in medicalimaging, there was a lot of work being
(04:43):
done on using handcrafted featuresand using all kinds of other modeling
techniques, but it was not scalable.
And just using lots of data, abstract feature representation,
that all sort of made sense to me.
Got it.
Great.
So, we want to actually dig into someof your work in AI for pathology.
(05:05):
And I honestly don't know where to start.
This is a huge compliment to you.
I think you've been justamazingly productive.
When I think of someone who's reallybeen at the forefront of AI for pathology,
you know, you're the first personthat comes to mind. And you've just
done this amazing, amazing amountof work for the past few years.
(05:26):
And again, this is a hard choice aboutlike, which paper of yours to start
with, but maybe I can start with
one that, out of personal interest,caught my eye a few years ago and
it's gotten a lot of attention that Ithink has also inspired a lot of folks.
And this is your paper that, andthis is another question that we have
to ask you at some point, which ishow you come up with these names,
(05:46):
but this is your paper on CLAM.
The title of the paper is "Data efficient and weakly
supervised computationalpathology on whole-slide images."
So, I think that title'sactually pretty informative.
This was published in NatureBiomedical Engineering a few years
ago, back in 2021, I believe.
Maybe we could start with that paper.
Could you tell us about like, first ofall, tell us what that title means, what
(06:06):
you were trying to do there, and maybealso take us back and give us a little
bit of the backstory on the paper itself.
Yeah, so, when I got started in thisfield, making some of these methods
data efficient was a big problem.
Lemme just give a littlebit of a background here.
So, glass slides, pathology glass slides.
When they're digitized, theyturn into gigapixel images.
(06:27):
So, these images are very, very largeand a single patient case can have
anywhere from a single slide to up to,you know, a couple of hundred slides
depending on what kind of case it is.
And the only label that you havecorresponding this entire case, a
multitude of very, very large gigapixelimages, they're almost like satellite
images, is the pathology report.
So, the relationship between your data andyour label is incredibly weak, and when
(06:52):
you have lots of images and single labels,
what comes to mind from a machinelearning point of view is that yes,
well, we can use multiple instance learning. But you know,
multiple instance learning can be inherently very data inefficient in
this case, because a very small regionfrom the gigapixel image can update
the weights and biases of the model.
And there were some work done beforeour 2021 Nature BME article showing
(07:15):
that you needed about 10,000 wholeslide images corresponding a single
class to train a model that would beequivalent to pixel level annotations.
So, data efficiency in this area was ahuge problem. And what my group inherently
does is that we develop solutions andmethods, computational methods that would
solve very targeted clinical problems.
(07:37):
So, this is a study that we didmore on the methods side, or
we try to solve this problem.
It's just being really focusedon how can you make this form
of supervised learning
more data efficient? And we useda number of different bells and
whistles at the time that were usedby conventional machine learning, like
attention, and models that were trainedon ImageNet, and just extracting features
(07:59):
from those models to make the setupinherently more, more data efficient.
But I think the really catalyzingfactor and why this article is so
popular is that we made all thecode and models and everything
corresponding to this publicly available.
So, people have really gone to townusing this tool for every major
organ system, every disease model,all the way up to forensics, to
(08:21):
wherever you have pathology images,
it's, it's been applied.
And—. Were you surprised by any of the uses of the model?
Yeah, we've been surprised thatnumber of times. People have used
it for plant pathology, things thatI just would've never imagined.
And at the time, the backstory isthat the real reason we did this
(08:41):
study, because I was very interestedin cancers of unknown primary.
So, when cancer metastasizes,it often becomes unclear
where it may have originated.
And identifying the primary originof tumor is really important because
most drugs are primary tumor specific.And these patients can't go in a clinical
trial because most trials would requirethat the primary be already identified.
(09:03):
So, we, we wanted to solve that problemand it's like a highly imbalanced,
multi-class multitask kind of a problem.
But there was no good way to solve itusing these very large gigapixel images
where the number of samples per patientcan vary and the number of cases we
have per patient is heavily imbalanced.
So, that's why we developed CLAM.And then we applied that to this
(09:27):
specific problem in cancer ofunknown primary, and that's a,
that's also a 2021 Nature article.
And we call it TOAD.
This is TOAD, right?
TOAD, yeah.
Yeah.
Okay.
So, how do you, I gotta ask now,how do you come up with the names?
So, it, it became just a theme.
So, so, I came up withCLAM 'cause I like— Okay.
—eating them.
Right.
So, okay.
So, so, so, so, it's, it'sone of my favorite foods.
(09:50):
So, I just came up with CLAM.
And it also made sense because it's,clustering constrained attention,
multiple instance learning. Nice.
But that sort of inspired my studentsto start coming up with names.
In the beginning it was all aquatic themes.
And one of my Ph.D. students who was from the big program, Richard,
(10:11):
he eventually created an internal packagethat we were using for quite some time.
It was called Fishing Rod.
So, you use the Fishing Rod tofish all these aquatic species.
Amazing.
Amazing.
I think you said something thatI thought was really important.
So, I just wanna dig into a littlebit, which is that you think part
of the reason that CLAM took off,and I think this has actually been a
(10:32):
theme in a lot of your work, right?
But one of the reasons youattributed to CLAM taking off was
releasing the model. Making it opensource. Making it available, making
it easy for people to build off of.
And I think this has been a, you know,I just, I'm immediately drawn to the
comparison with, what David Ouyangdoes for AI, for cardiology. He was
a guest we had on a couple months ago.
(10:52):
And, you know, a coupleother folks who've really
emphasized the importance of it.
And I don't think we can say thisenough because I think it's just,
it really changes the trajectory of
the paper afterwards, if people canreally build off of it easily, and if
they can, use it to do their own work.
And so, I'll commend you on that, but Ialso think it's worth just maybe talking
(11:14):
about that a little bit more, right?
Like, I think there's a lot ofthings that are attached to it.
You have, you're naming it right?
You're sort of, this is the wrong word,but you're making it an entity, right?
Productizing, that'sthe wrong word for it.
But you're making it sort of a thing.
You're releasing resources that accompanythat paper, not just sort of ending
at the science or the inference orthe findings, which are fascinating.
(11:36):
They're interesting, but you're alsomaking it sort of a engineering project
that people can then build off of.
So, maybe you could talk about that,like what are the ingredients here?
I think you've done this a lot.
You know, what are, what'sreally important to emphasize for
someone who wants to emulate that?
And, and, and maybe
if you could comment on the extent towhich you get institutional pushback or
support on this, because my experiencehas been that, like, if I train a
(11:59):
model on hospital data, the hospitalgets very nervous about potentially
releasing the model, not even the data.
So, I don't, I, I, if you haveany, uh, experience with that,
I'd love to hear about that, too.
Oh yeah, absolutely.
So, with CLAM, it was mostly the codebase because, we had some hospital
test data, but the models were largelytrained on the TCGA at the time.
(12:21):
I think that making that entire code base
publicly available, and it's beena common theme in all of our work.
We always make the code, everythingas much as we can and as much as is
allowed publicly available has beena big accelerator for us. And it has
led to all kinds of improvements.
Things that I had never imagined becauseonce we made CLAM publicly available
(12:42):
everyone from the entire communitystarted contributing and giving
us feedback as to what we can improve.
And of course, we do a lot of improvementinternally as students learn what is
the best way to do what over time.
But we also had this community feedbackthat led to a lot of systematization,
computational improvement, just ideasfor how we can improve the entire
(13:03):
pipeline and, and, and scale it.
And to your question, Andy, so, absolutely.
It was sort of an internal battle to try to make some of the
larger foundation models thatwe had built publicly available.
We didn't face any resistance. It'sjust a matter of understanding.
So, we had the innovations office andcorresponding legal people involved and
(13:25):
trying to understand what are, what isit that we are trying to make public.
And once they understood that the datais not being made public. It's a model
that's trained in a self-supervisedmanner, no identifiable labels were
used in the, in the training.
And there, were by the time wewere making the first self-supervised
large scale model public, there wereother examples from a lot of other
(13:48):
institutions, including from Stanford,that had already been made public.
And then they did a lot of digging and trying to understand what kind
of license should be used for this.
But eventually we did get the permissionsto make the make the models public.
Awesome.
I'd like to now hop to another one ofyour papers and continue the shameless
(14:08):
flattery here that Raj started andsay that like, there's like two
constancies in my Twitter feed.
One is that Elon Musk has done orposted something crazy, and two
is that Faisal has posted anotherNature paper on AI for pathology.
So, so, thanks for being that like steadyrock in my Twitter timeline.
Faisal, so this paper I think is
interesting
(14:29):
not just because the work was firstrate, but also because of the model it
suggests for AI going forward in medicine.
And so, the paper is "A multimodalgenerative AI copilot for human pathology."
And so, this was your PathChat paper.
So, one, I'd love to like for youto tell the listeners what PathChat
is, what it does, and whether or notco-pilot is a mental model that we should
(14:52):
have for AI and medicine going forward.
Yeah, absolutely.
So, the story of this study is thatI often call it an accidental study.
So, we started by, so, we'resolving all these entrusting,
supervised problems, and they'rebecoming more and more complex.
We started by solving very small, supervised classification
(15:14):
problems. And then they became more complex for solving multi-class,
multitask problems still, verysupervised. And, but the backbone behind
it is that because these images arelarge, we pre-extract features using a
ResNet that's just trained on ImageNet.
And then everything that was happening inconventional computer vision and machine
learning in general with, with selfsupervised learning, it became clear.
That you can find just amazing applicationpathology where you have all these rare
(15:38):
diseases, clinical trials, situationswhere you don't have enough data
available for, for supervised learning.
And we started to train large selfsupervised models, maximizing for
diversity, trying to collect every known
human pathology indication that wouldexist in our archives. Because it was
established using a number of differentstudies, including DINOv2 from Meta
(16:00):
showing that the diversity of data mattersway more than the quantity of data.
So, we max, try to maximize for diversityand we published two foundation
models in Nature Medicine and
once we had that, it obviously accelerateda lot of research we're doing and
just building more supervised models.
But another thing that we could dowith those self-supervised models is
because it can extract rich featurerepresentations now from pathology
(16:23):
images, is that we could, if we haveenough instructions, we could train a
multi-model, large language model, right?
So,
the goal was that can we have asingle multimodal, large language model
that can cater to all of human pathology?
And there's a lot of philosophy behindit as to why we think that's possible.
Like, companies like OpenAI aretrying to build a single multimodal,
large language model that willcater to human knowledge and
(16:46):
pathology is much more specific.
And we, if we have enough data
there's a high chance that you canconverge to a model that would do well
around everything around pathologies.
Just that data is locked upand it's not easily available.
And the other challenge was thatnow we have to go in the opposite
direction that pathology reportsdon't have enough morphologic details.
(17:06):
And a perfect human pathology chatbotor a co-pilot should be able to answer
questions at any magnification level.
It should be able to answer questionsfor specific regions in an image, for a
single image, or a multitude of images,
to be really, really useful.
So, we had to collect a lot of datamanually, but we also made use of
data that was used for teachingpurposes at the Brigham and MGH.
(17:28):
So, we have a lot of colleaguesand friends who contributed data
to this, but it was also a verytargeted data collection effort,
including a lot of other institutions.
And then we eventually had avery large instruction set.
We trained the multimodal largelanguage model, and we eventually
had a chatbot that we could usefor any pathology indication.
And then the assessment of the model wasmore difficult than building it because
(17:51):
we already had the foundation model and wespent a lot of time collecting the data.
We built
the model, and this entire processtook about a year. But it took
another year to just come up with thatevaluation paradigm, that would be
rigorous enough, took a lot of inspirationfor how, for what was happening in the
large language model research, both formedicine and in other critical areas.
(18:11):
And yeah, and then it waspublished last year around June.
We have a lot of demos aroundit, publicly available.
So, maybe I just wannaconnect a couple dots there
based on what you said. So, like,in large language model development,
there's often this very expensive thingcalled pre-training where you create
the base model and that creates amodel that has seen a lot of data and
can do many things, but it doesn't do
(18:33):
many things well.
And specifically, one ofthe things that it doesn't do
well is interact with people.
Yeah.
And so, after this large foundationmodel has been created, there's this
set of techniques called post trainingthat's often applied to the base model.
Some of it is supervised fine tuningor instruction. Fine tuning, like
you mentioned, where you actuallyteach the model to follow instructions.
(18:53):
Sometimes you do reinforcementlearning from human feedback.
There's this whole kind of like dark art—
Yeah.
—to making these models useful.
And it sounds to me like what you weredoing was sort of a one-for-one analog of
going from base GPT to ChatGPT whereyou took this very powerful foundation
model, but made it usable for people.
Is that a fair characterization?
Oh yeah, absolutely.
(19:14):
So, I often say that we didtwo stages, but not three.
So, a typical three stage processwould be that you have the strong backbone
model, you do instruction tuning on topof it, and then you do reinforcement-
learning-based fine tuning whereyou get a lot of human user feedback.
The model can generate many responses andthe human would say, which one is better?
And then you use that to, tofurther—. I assume that Nature
paper will be coming out this June.
I'm sure that, that's somethingthat you're working on now.
(19:35):
We're working on it.
It's, it's tedious and difficult because—.One Nature paper a year or I'm
not impressed anymore, Faisal, youset a really high, a high standard.
Yeah.
And so, just to, okay, so, uh, I believethat recently PathChat has become
the object of commercialization fromyour lab, that there's a spin out
(19:56):
around that, and that you received anFDA breakthrough designation for it.
Can you tell us more about that?
Yeah, absolutely.
So, two of my Ph.D. students, Max Liu
from MITE and Richard Chen fromfrom the Harvard Big Program, they had
decided early on that they wanted tostart a company once they graduate.
And PathChat seemed like a goodopportunity because it became
(20:16):
clear once we sort of announced it when it was
a preprint, that there was justa lot of interest. And I guess
it was also around when generativeAI, I mean it still does,
a lot of hype around it.
And it is truly useful in the sense thatit could be great for training, but also
it has the capability to impact the entirehorizontal of the pathology workflow.
(20:37):
Right?
Because it, it's a chatbot,but you can also constrain the
outputs to do other things.
Like it can become a universal triagetool, or it can look at a case and
suggest what ancillary tests and IHCs toorder, and those can already be ordered
before a pathologist is looking at it,or it can write the pathology report.
So, so, all of these auxiliarythings it can do can have a
(20:57):
lot of impact in pathology.
So, Richard and Max startedthe company together with some
of my other colleagues, and thenwe were thinking that what's the
best regulatory pathway for this?
And what my thinking and thinking ofmy colleagues in general is, how does
digital pathology get universal adoption?
Because the digitization rate aroundthe country is just around 4%.
(21:22):
It's very limited, and there area number of reasons for this.
It's expensive to digitize these images.
It's expensive to store them, but itobviously makes sense to everyone because
the research benefits down the line are just enormous because we
haven't discussed the discovery aspectaround these, images, but we think that
if there is sort of a killerapp, a killer AI tool, that would
(21:42):
make everyone's life easy peoplewould go digital very quickly.
So, we think that a universal triagedevice for pathology could drive a
lot of digital pathology adoption.
And the reason is that pathologistswould often look at many cases.
And within each case there wouldbe lots of slides and only a
fraction of them might be positive.
And if the device can tell you whichones are negative, it can make life
(22:04):
easy for a lot of people.
And there's a good predicate to this.
A lot of people think that AI forpathology devices have recently begun
to be approved by the FDA, but therewas one that was approved in 1998.
It was called Auto Pap.
So, it was just a camera attached to amotor, taking images from a slide and
doing a line profile through the cells.
(22:24):
It was for pap smear detection,and that was around when
there were a lot of pap smears tobe, to be analyzed, and there was a
shortage of of experts to do so.
So, the FDA approved this.
And it has something calledan NSR or no second review.
So, this meant that all the negativeslides could be screened or triaged
(22:44):
out and the experts would only haveto look at about 10% of the negatives.
So, that's one of the ideas and PathChatcan inherently be a very good triage tool.
And it's also
an assistive tool.
And the reason we wanted to getbreakthrough device designation is because
this is a relatively new technology andbreakthrough device designation gives
(23:04):
you access to the FDA in the sense thatthey would meet with you regularly and
design like a pathway towards approval.
And it's often the best course ofaction for new technologies and devices.
And generative AI is relatively new.
I mean, there are all these like openquestions as to how do you get a device
approved through the FDA that can make adiagnosis corresponding every indication.
(23:27):
So, that was the reason to get thebreakthrough device designation.
The FDA requires evidence around theefficacy of the device and that it's
truly sort of a quote unquote breakthroughto give that kind of approval.
So, that's what we essentially did.
And it was granted earlier this year.
Super interesting.
First of all, shout out RichardChen, who was an intern with me.
The first intern I think I eversupervised when I was a postdoc.
(23:50):
So, it sounds like he's goneon to spectacular things.
Yeah.
Second of all, one of the things thatI've learned from being in this space
and certainly from this podcast, isthat technology development is hard.
Commercialization isoften much harder still.
And so, how do you think aboutbusiness models around this?
Is there gonna be a reimbursement model?
(24:11):
Is this something like, I guess like howdo you think about sort of market value
or what even the market is in this case?
Yeah, that's a great questionand I know Richard has been
thinking about it a lot.
And I definitely agree with you.
Richard is one of a kind.
He was our superstar graduatestudent and did really, really
well, in the lab and is sort of
addicted to making digital pathologyand computational pathology just be
(24:33):
absorbed in the clinic, and he reallywants to change the clinical paradigm.
So, it's a really big passion of his.
I think that there's some benefithere from the predicate we have in
cytology that has been reimbursedfor a very, very long time.
So, reimbursement is,
no doubt, a challenge for, it's achallenge in AI for health in general,
and it sort of trickles down into pathology, but it might not be that
(24:55):
big of a challenge for a triage device.
The bigger challenge that we see isjust digital pathology adoption, how
quickly it would happen, and how muchof a trigger would FDA approval for
a primary triage would be. So,besides Auto Pap, all other more recent FDA
approvals for AI for pathology toolshave all been for a secondary analysis.
(25:18):
So, pathologist first looks atit, and then the machine learning
would essentially confirm.
So, if we do get approval for thisdown the line, it would be for
primary triage at the slide level.
And then we hope that that would sort ofdrive more digital pathology adoption.
Cool.
Awesome.
Thanks.
Okay.
I think we're ready to moveto the lightning ground.
(25:46):
Are you ready?
Yeah.
Yeah.
Ready.
Andy, did you wanna give us the introand the rules of the lightning round?
Sure.
In the lightning round we'llask you a series of questions.
It's up to you to decide howserious or non-serious to take them.
Short answers are rewarded handsomely,and long answers are punished viciously.
So, those are the rulesof the lightning round.
(26:06):
So, Faisal do you accept?
Do you accept the task?
Absolutely.
So, this one is a little bit of a,I would say, pet peeve of mine.
In that I have irrationally strongopinions on it, but in your opinion, do
you think that AI should be explainable?
To a degree.
I love, uh, one of the FDApresentations showing something.
(26:28):
It said that AI gives hints, andI completely agree with that, but
I don't think we should obsessover interpretability too much.
The performance evaluation, theclinical trials, all of that is,
is much more stronger evidence.
There's, there's a lot of otherthings in health care and medicine
that we can't really explain.
I agree with that
100%.
Cool.
This is just that, that answerwas music to Andy's ears.
(26:50):
I know.
So, Faisal, nextlightning round question.
What's your favorite novel?
Oh, uh, Norwegian Wood by Haruki Murakami.
Wow.
Nice.
Deep cut.
If you weren't a professor atan academic medical center, what
(27:11):
would you be doing with your life?
Oh, so, if I was not a professor at anacademic medical center? Or more generally
doing health care, like medical AI.
So, if in the counterfactual world, Faisalwas doing something completely different,
what would counterfactual Faisal be doing?
Oh, I'll be writing.
Oh, okay.
Novels, like the onethat you just mentioned?
(27:31):
Yeah, yeah.
Something like that.
have you written before? Have youwritten fiction or nonfiction before?
No, but that's what Iwanted to do as a kid.
Oh, wow.
Yeah.
Faisal, will AI and medicinebe driven more by computer
scientists or by clinicians?
So, I think, it'll be a balance.
So, if I just look at what I'velearned over the past decade, some of
(27:53):
the best projects in my lab have comeabout when computational trainees and
clinical trainees partner together.
And I've been very lucky to havegreat graduate students from
amazing Harvard and MIT programs,but also just great residents
and fellows working in the group.
And some
were just amazing withPh.D.s in computer science.
(28:14):
And I've also gone throughentire clinical trainings.
When they work together,that's when the magic happens.
Yeah.
Cool.
Alright.
Last lightning round question.
It's one that we ask most of our guestsand we always get interesting answers.
If you could have dinner with oneperson, dead or alive, who would it be?
Ooh.
(28:34):
Wow.
Uh, that's a, that's a tough one.
Tesla.
Oh, Nikola Tesla.
Yeah.
Nikola Tesla.
I mean, my background isin electrical engineering.
Just really inspired by all theout of the box thinking, but
still everything that made sense.
I wish it was possible.
Yeah.
And unfortunately overshadowed byEdison in many regards, despite
(28:55):
being his equal or greater, so. Yeah.
Yeah.
Cool.
Awesome.
Well, Faisal, you, yousurvived the lightning round.
Congratulations.
Thank you.
Alright, so, we have a couplebig picture questions to
throw at you before we wrap up.
Again, I think we've touched onthis a little bit throughout the
episode, but do you have a, like a,a unique ability to select problems
(29:16):
that have the dual properties ofbeing interesting, but also not
stuff that everyone else is workingon, and therefore you're able to do
meaningful work without momentum chasing.
And so, what is your method forselecting interesting problems
in such a fast-moving field?
(29:37):
So, I pick problems for which there'sno good solution right now, right?
So, I think that of course we can usemachine learning to replicate what humans
do and maybe do a little bit better,reduce intra observer variability.
All of that is great,but I just feel that
(29:57):
we as machine learning for healthresearchers can make a bigger impact if
we target problems that for which there'ssort of no good solution right now.
Right?
So, so, it was reflected in my postdoc,I worked on trying to see if we can
use machine learning to predict depth from2D conventional and endoscopes, right?
So, so, we generate a lot of syntheticdata and try to solve that problem.
(30:21):
If we're able to do that,you can essentially convert
any conventional 2D endoscope intoa 2.5D or a 3D endoscope.
And the same is reflectedin my more recent works over
cancers of unknown primary.
There, there are a numberof different solutions, but
there's no good solution.
And these cases are just, it takes sucha long time to diagnose these cases.
(30:44):
And the other large study wedid, it was back in 2022 for endo
myocardial biopsy assessments.
There's just huge intraobserver variability.
It has major issues with the downstreamtreatment, so, just problems for
which there is no clear solution.
People are much more amenable toadapt those models into clinical
(31:06):
practice versus trying to replicateor just build assistive tools.
Awesome.
Faisal, this is our last questionfor you, and maybe I can get you
to give us your future vision forhow medical AI is gonna evolve.
And specifically, I think we'd beinterested in your taking.
I think we've heard a lot about howyou select interesting problems,
(31:28):
how you've designed medicalAI applications for pathology.
But maybe you can reflect on and projectfor specifically translation into the
clinic, how you see medical AI shaping,reshaping, changing, not really changing
medicine over the next five to 10 years.
(31:49):
And, I think it would also be veryinteresting for you to tell us about
what you think are the key bottlenecks.
Are they social?
Are they technical?
Are they regulatory?
What are the key bottlenecks tomoving AI from where it is right now
to a greater role in clinical care?
Yeah, I think that there are twolines of development that would
(32:11):
happen over the next five years or so.
But and it always happens soonerthan I think so, so, it might even
happen in the next few years.
One of those directions is thatI think that we'll soon have
patient level models, right?
So, we're seeing that all these self-supervised models in individual domains
are becoming better and better atrepresenting those particular modalities.
(32:32):
And we're seeing some earlier flavorof multimodal, so, quote unquote
foundation models, where contrastingwith another modality improves
representation for all modalities.
We'll see that done in a muchmore holistic way, where the
patient's entire medical record.
It could be featurized into a singlefeature vector, and it would
have a temporal component to it.
So, if a patient who's on immunotherapyhas a toxicity event over the weekend
(32:54):
their feature vector has now changed.
And, and then all the downstreamvariables would reflect that.
And that would do essentially two things,would really enable a lot of early
detection, early diagnosis, and it canbe happening on the fly all the time.
And I think that achieving thisis very much possible today.
it's only a matter oftime before this happens.
Once this happens, we will alsohave the ability to use entire
(33:17):
clinical grade health care systemscale data for discovery, right?
Because if you look throughout thehistory of medicine, how do people come
up with new diseases, new subtypes?Is that someone goes in, looks at lots
of data and says that, well, this
particular, morphologic phenotype onan image or all of this categorization
correlates with outcome, and this issubtype A and subtype B and disease A and
(33:38):
disease B, and we have the opportunityto do that automatically on its own,
almost continuously, as the ology ofsome of these diseases evolves over time.
So, it'll become like automatic
discovery engines.
That's one thing.
And then the other thing I think that willhappen, and it's also in the short term,
is that with all the agents and agentworkflows, computational biology as we
(34:02):
know, it would just fundamentally change.Because what computational biologists
do today, where you have existing dataor new data that you have collected,
and you're often using existing toolsand you're patching together some code
that you write and using more tools,and sometimes you're developing newer
tools, would just be done on its own.
And a lot of hypothesis generationand testing and assessment
(34:22):
can also happen on its own.
So, I think those are the two big things that are, that I
think are just obviously coming.
So, just patient level foundationmodels that will enable both clinical
change as well as lots of discovery and then agent workflows that would,
that would just accelerate all kindsof computational biology research.
(34:43):
Alright, well I think that'sa great note to end on.
Thank you so much forbeing on AI Grand Rounds.
Yeah.
Thank you so much for having me.
Thanks for coming, Faisal.
Thanks.
Thanks guys.
This copyrighted podcast from theMassachusetts Medical Society may
not be reproduced, distributed,or used for commercial purposes
without prior written permission ofthe Massachusetts Medical Society.
(35:07):
For information on reusing NEJM Grouppodcasts, please visit the permissions
and licensing page at the NEJM website.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Intentionally Disturbing
Join me on this podcast as I navigate the murky waters of human behavior, current events, and personal anecdotes through in-depth interviews with incredible people—all served with a generous helping of sarcasm and satire. After years as a forensic and clinical psychologist, I offer a unique interview style and a low tolerance for bullshit, quickly steering conversations toward depth and darkness. I honor the seriousness while also appreciating wit. I’m your guide through the twisted labyrinth of the human psyche, armed with dark humor and biting wit.