In this episode of NEJM AI Grand Rounds, guests Drs. Alan Karthikesalingam and Anil Palepu of Google walk co-hosts Raj Manrai and Andy Beam through the making and evaluation of AMIE, an AI system designed to conduct clinical conversations with patients. Alan and Anil explain how AMIE was trained using synthetic doctor-patient interactions generated by LLMs playing multiple roles—doctor, patient, critic, and moderator. They reveal how synthetic dialogue, guided by structured feedback and grounded in search, proved more effective than noisy real-world transcripts in building a model that could reason, ask questions, and show empathy. The discussion also covers what the “long tail” of medicine demands for building robust AI systems and how AMIE might one day augment real clinical workflows. Finally, Alan reflects on how AI has changed—and how it hasn’t—in the two years since he was last on the podcast.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:01):
It has been a consistent thread howpeople can interact with these systems.
Like even, I mean, we were talkingearlier about medical imaging and
I remember, you know, all this workfrom concept bottlenecks to kind of,
like activation-based heat maps.
You know, that lovely workactually of other networks
analyzing how classifiers work.
Like why is a zebra, a zebra?
Well, actually it turns out it's becauseit, it's activating all the weights
(00:21):
for being stripey and there's otheractivations going on for being horse-like.
And that may be much moreuseful than a sort of blotchy,
pixel-wise activation map.
And I think when you kind of move intothis era that we are in now, systems which
can actually go away and do work, likethey can work for potentially even days
before coming back to you, bringing backvast swaths of information, I think it's
gonna be critical to sort of optimizeand think about these things properly.
(00:45):
You know, again, look, maybe it'smy bias as a doctor, but I think
medicine is the most humane
of things, right?
It's the most essential human condition,and so our ability to actually
interact with these systems properlyis gonna be very, very important.
Welcome to another episodeof NEJM AI Grand Rounds.
(01:07):
I'm your co-host Raj Manrai.
Today, Andy and I are excited tohave two guests on the podcast,
Dr. Alan Karthikesalingam of Google DeepMind,and Dr. Anil Palepu of Google.
Listeners of the podcastwill remember Alan.
He's making his second appearanceon AI Grand Rounds, and Anil
joins us for the first time.
Together they told us about Google'sarticulate medical intelligence explorer,
(01:29):
often referred to as either AMIE or AMIE.
AMIE is an AI system that's designedto directly interact with patients.
Alan and Anil took us through theproject from ideation to building
it to some of the evaluationstudies that are happening today.
It was also a great chance to reflectwith Alan on how AI has changed and
how it hasn't since he was last onthe podcast about two years ago.
(01:51):
Alan has this special ability to distillcomplex technical and clinical ideas
into their essence, and he really isa wonderful example of a physician
who's really become a leader in AI.
The NEJM AI Grand Rounds Podcastis brought to you by Microsoft,
Viz.ai, Lyric, and Elevance Health.
(02:14):
We thank them for their support.
And with that, we bring youour conversation with doctors
Alan Karthikesalingam and Anil Palepu.
Alright, well, Anil and Alan,welcome to AI Grand Rounds.
We're excited to have you today.
Yeah, thanks for having us.
Yeah.
Thanks for having us, Andy.
With the exception of Zak Kohane,
(02:35):
I think that Alan is the first repeatguest that we've had on the podcast.
So great to have you back, Alan.
I think that's right.
Yeah, that's an honor.
Way to go to
intimidate me at the start.
So as a result, Alan, we've actuallyalready asked you the following question,
so I'm gonna pose this question to Anil.
Anil, this is a question thatwe always get started with,
that we ask all of our guests.
(02:55):
Could you tell us about the trainingprocedure for your own neural network?
How did you get interested in artificialintelligence and what data and experiences
led you to where you are today?
Yeah, I did my undergrad in 2016.
That's when I started.
At the time, I think I had.
Very little idea of what I wanted to do.
I was interested in medicine,but I definitely did not
(03:18):
know that I wanted to do AI.
I don't think I knew whatAI was at that point.
I basically just knew that I didn'treally think I could be a doctor.
I was a bit too lazy.
I didn't think I could studyfor the MCAT or do all the
steps that are needed for that.
So, I was like maybe doing somethingelse like perhaps computational
would be a bit easier for me.
And that's kind of how I got started.
It's not like a glorious story of knowingI had a dream and reaching it, but as I
(03:43):
started to do research, I really fell inlove with the type of work we were doing.
And I found that I had been doing likea lot of projects alongside clinicians
and driven by a clinical need.
And I was really excited that
it felt like these people allhad some serious problems.
What really struck me was everyone wassuper unsatisfied with the status quo,
(04:04):
and there were so many things to improve.And me, as an undergrad, I actually felt
like I could apply the things I learnedin class. Like, me learning like about
random forests two weeks ago, and then
applying that to like a real problem andactually seemingly making a difference
in the workflows of these clinicianswas like crazy inspiring to me.
And that led me to be like, okay, Iwanna take this to the next level.
(04:27):
I wanna do my Ph.D. and keepdiving down in this path.
And so I did that.
I applied for my Ph.D.
I went to MIT and had the pleasureof working with an amazing advisor
there who is not paying me to saythis, but he's on this call right now.
And he taught me a lot about deeplearning and got me really excited,
particularly in the tech space for him.
That's when I joined Google.
(04:48):
Met some amazing coworkers there,Alan included, and still there today.
Keep learning from every day and reallyexcited about the mission we're pushing.
Cool before we come back to your worldclass graduate training and mentorship,
'Cause we're gonna come backand spend a lot of time on that.
I feel like this is gonnabe a recurring theme.
Yeah, it's gonna be a recurring theme.
Yeah.
So, you're yet another person fromthis HST program at Harvard and MIT.
(05:10):
So, this isn't like something thatwe've planned, but it just seems
to be that this program produces alot of folks at this intersection.
However, before you got inthis, could you take us back to
early days of Anil, when you werestill like much more pluripotent
before you had even thought aboutcomputer science and engineering.
What did you enjoy growing up andhow did you end up in a biomedical
engineering program in the first place?
As a kid, like I really looked upto, I think my family, particularly
(05:34):
my brother, I have an older brother,and he always had a very strong
vision of what he wanted to do.
And he knew he wanted to be adoctor and everyone could see it,
he really had that drive in him.
And I was like, he knows what he's doing,he's doing all the work to figure out
what's important and what to work on.
So maybe I'll do that.
And again, like I said, I think Iknew what my skillset was, I knew
(05:54):
what my limitations were, so I waslike, I don't think I could actually
do the work of seeing patientsand have that on day-to-day.
Like, I wanna sit on my couchand watch TV while I work.
That doesn't seem possible ifyou're seeing patients all day.
So, I found some path that was,I think very inspired by him.
I think I've also certainly changeda lot as well, especially as I've
(06:15):
gotten deeper into the research andI think I'm a little bit more driven,
at this point than I started out.
Awesome.
Thanks for that, Anil.
So, I think we're gonna hop in and Alanmaybe just quickly so that we don't
shortchange you, do you wanna hop back onand just quickly introduce yourself again?
Yeah, of course.
Thanks again for having me.
So, I'm Alan. I'm a clinician and aresearcher at DeepMind, a company
(06:35):
I joined kind of about eight yearsago now. And I've always worked on
AI for biomedicine, doing stintsbasically at DeepMind itself,
also in different teams in Google.
I'm very lucky to work withAnil for a good chunk of that
on some very exciting work.
Awesome, thanks.
So now we're gonna transition andtalk about some of the projects that
you both have worked on together.
And I'll preface this by saying the firstpaper that we're gonna discuss, again,
(07:00):
Zak Kohane mentioned this in our scotchinfused end-of-the-year round table.
Did you
guys, wait just as a, didyou guys listen to that one?
So it's a good one.
We discussed AMIE, quite a bit,so I think, you'll enjoy that.
But anyway, go ahead.
Yeah.
Yeah.
And one of the things we discussed isactually the correct pronunciation.
Is it AMIE or is it AMIE?
I think this is, uh, is it potato, or is it?
(07:25):
Yeah, ask three doctors.
You'll get three answers.
One of the things about a transatlanticteam is I think we find, like, almost
every way to pronounce almost every word.
So, we'll leave that one a mystery for you.
Okay.
Well, uh, in my slightly southerntwang, I'm gonna say AMIE here
forward and Raj can go with the morecosmopolitan pronunciation of AMIE.
Okay, sounds
(07:46):
good.
Um, so anyway, we had this end ofthe year episode with Zak, and one
of the things that we do is round uppapers that have been published at
NEJM AI, and then we talk about papersthat have been published elsewhere.
And Zak and I actually both pointedto the paper that we're gonna talk
about next as one of the papersthat we thought was most impactful,
that was not published at NEJM AI.
(08:07):
And so, the paper is "Towards conversationaldiagnostic artificial intelligence."
And instead of doing you both thedisservice of butchering the explanation
set up and context how about we goto Anil and you give us the technical
overview and the technical setup, andthen we'll go back to Alan and discuss
some of the clinical aspects of it.
Yeah.
So, this paper "Towards conversationaldiagnostic AI," essentially what we're
(08:28):
dealing with is up until this point, therewas a lot of work showing essentially
that, these LLM based systems do exhibitsome quality medical reasoning. They were
tested on these benchmarks like MedQA,and, you know, at least in like these
multiple-choice settings, they performedquite well and often superhuman.
That being said, there was verylittle work investigating how well
(08:51):
these systems can actually elicitthis information from patients.
And so, this is a setting where ratherthan having the entire picture provided
upfront and an LLM simply interpreting it.
In a real clinic, a clinician willhave to actually talk to a patient, get
the information from them, do so in away where they're exhibiting empathy
(09:12):
and quality conversational skillsfor history taking, and ultimately
still come up with the correct planand differential for the patient.
So, this was what we really wanted to test.
Alan and others really set a really greatvision for how we would test this, which
is through something called an OSCE orObjective Structured Clinical Examination.
And so, we modeled our study aroundtext-based OSCE and we tested whether
(09:37):
our system that we developed couldperform this kind of consultation
with patient actors at the levelof primary care physicians.
And so that's what we did for the study.
In terms of the actual modeling,at the time we were working with
PaLM-based models and reallythe key innovation here was
we were trying to simulate conversationsbetween doctors and patients and use that
(10:01):
to actually train a model to emulate therole of a doctor in those conversations.
Can I hop in here?
'cause I remember you actuallypresented this at a lab meeting one
time, and there was a couple technicalaspects that I thought were just,
like, both surprising and interesting.
If I remember correctly, you have this
team of PaLM models, one is acting asthe doctor, one's acting as the patient.
(10:24):
And you use this ensemble of LargeLanguage Models to synthetically generate
data, and then you train on that dataand the model keeps getting better.
And I remember at the time I was deeplyskeptical of pulling yourself up by
your own bootstraps aspect of whatpeople are talking about in synthetic
data, but this seemed to be a prettycompelling example of where you could
have the model generate syntheticdata and it get better at the task
(10:45):
that you're trying to get it to learn.
So, maybe could you talk a littlebit about, like, that aspect?
Was that surprising to you or have youseen other examples of this sort of
synthetic data trick working in practice?
Yeah, so our setup, as you describedreally well, is we had this multi-agent
system where we had essentially wehad that doctor and that patient each
played by PaLM at the time and theywere interacting with each other.
(11:07):
And we also had a separate agent, amoderator, which was just basically
checking whether conversation was done.
And really the criticalagent was this critic agent,
which, after an entire conversationwas complete, it would review the
conversation, identify things thatthe doctor did right or wrong,
and pass that to the doctor.
The doctor got to trythe conversation again.
And in doing so, we were able to guidethe conversation to words, what was better
(11:32):
doctor behavior according to how we knewwe'd be evaluated and what the optimal
plan for this patient would've been.
The other aspect of this that I thinkis injecting new knowledge is the fact
that our conditions, our vignettes,were actually conditioned on search.
And so essentially here we're ableto bring in some kind of grounding
(11:54):
that is able to be providedto the model during training.
I think the other important point tonote is that these PaLM-based models
initially just like simply could not,if you tried to prompt them to do this,
they would not hold a conversation.
Certainly not over this many turns.
And so, I think a lot of our gainswere really in being able to hold
(12:14):
that conversation, like the style of
that diagnostic dialogue.
Whereas like today, we don't actually needto always do significant post training.
Like in our most recent work, we'reactually using the base model for
most of the heavy lifting there.
So I think that is somethingthat has also just changed over
time in terms of necessity.
So cool.
(12:35):
A couple other things now that thisis all loaded into my context window.
I remember the really surprisingpart of this is that you had all
of this presumably expensive humanOSCE data of actual conversations.
And one of the conclusions I rememberreading in the paper was that actually
the synthetic data ended up beingmore valuable than the human data.
Is that a correct recollection?
(12:56):
I guess valuable in what sense?
In that just training on thehuman data was insufficient.
And you really needed the synthetic datato be able to get the model to learn.
Oh yeah.
So, we had a human, like we hadtranscripts, which we were training
on, and I think, yeah, there were manylimitations with those transcripts.
I'm sure if we have, like,
(13:16):
human data in the exact style ofthe kind of consultations we wanted,
that would be a different story.
But what we're dealing withis very noisy transcripts that
maybe didn't cover the range ofconditions we wanted to cover.
And so, I think in that sense,by synthetically creating them,
we're able to actually tailorwhat we're providing to the model.
But I think a large part of thatwas A, like that it was an audio
(13:39):
transcript and B, that we were able toflexibly choose the set of conditions.
Yeah, to add a bit of color to this, youknow, you can imagine, I suppose it's
like one thing for imitation learningto have a worked example of exactly how
you would want an agent to behave itself.
It's quite another thing to be doingSFT on some transcripts, which are
(13:59):
literally just transcribed from
video consultations in which the providerand the patient might exchange a joke,
but about something that has visuallyjust sort of happened, like a bird
flew outside the window or something.
And there's a lot of what I just did,you know, a lot of, um, ahhh, but, you
know, this kind of thing faithfullytranscribed so that, plus I think Anil
(14:19):
makes a great point about the breadthand depth of conditions, and one of the
many beautiful things about medicineis that it is essentially a long tail.
You know, even if you take commonconditions, there's a kind of universe
of ways in which even the most commonor garden thing can actually present
differently in different people.
And so, the ability of utilizing self-play,I think one wonderful thing about that is
(14:42):
the ability to permute those situationsto bring search into the loop, which
means you're not relying on memory that'son information that like happened to be
encoded in at pre-training and to muchmore fully go beyond like the experience,
right, of what happens to have beenwritten down in any one data set.
Alan, that sort of reminds meof with image-based models,
the augmentations, right?
(15:04):
The sort of flips, and the permutations, andthe rotations, and blurring, and shearing,
other things that we do that are nowjust kind of standard practice, right?
You're sort of permuting all the ways topresent information to the model and then
trying to see what emerges out of itthat's more stable and more robust.
Yes.
Is that, is that fair?
Yeah.
I love that analogy.
So, you know, there was, there's been thiskind of arc over the last eight years in
(15:25):
which I think for, you know, I've alwayshad this kind of dream of systems like
AMIE, and it was one of the reasons whylike baffling my family, I left a very
lovely and amazing academicclinical practice, right?
And, like, a very nice lab andeverything to come to DeepMind.
And you know, there has been this arc fromkind of supervised learning and imitation
learning of these point tasks, producingthese systems that were useful for one
(15:47):
thing but fragile and not generalizable.
And not robust.
And in the era just precedingthe advent of these amazing LLMs,
we were doing a lot of work.
Trying to address those limitationsthrough techniques like their self
supervised learning especially, and weactually had a paper in Nature Medicine
on the use of synthetic data in theradiology context and the dermatology
(16:11):
context specifically for doing exactlywhat you were saying, but imagining,
you know, those augmentation regimes,but actually grounded in the domain.
And even there again, we found somepretty great impacts, including
non-trivial improvements to things likefairness in that context, and again,
I think for medicine, this becomesimportant because of the long tail.
(16:32):
So, when you think about all the ways inwhich a condition can present and all the
intersectional groups you actually haveto account for, it feels to me like an
almost impossible challenge to try and getthat distribution represented properly.
Under like a supervised learningregime or under like a regime of real
data that is going to be difficult.
(16:53):
It's kind of like a car that hasto learn how to drive by memorizing
every possible road situation.
It just doesn't seem feasible.
And so being able to leverage selfplay, synthetic data regimes, the
intelligent acquisition or likemodification of those data, like
AI, that feels like a very powerfulparadigm for medicine and life sciences.
Can I ask a technicalquestion on that, Alan?
(17:13):
And feel free to
take a pass on this.
So, where does the information come from?
So, you have this long tail of conditions,you're pulling data out of the weights
of the model, and so presumably,like, no new information is created.
I understand that, like, supervised finetuning post-training argument where now
you're getting the model to be in theright shape of things, meaning that it's
(17:34):
having this like turn-by-turn dialogue.
But what's your senseof this sort of, like,
added the information gain from doingthis because in, in self-play, like
Go, the model can actually play gamesthat have never been played before
because you have this perfect simulatorof Go, but in the case of generating
synthetic data, you're drawing fromsome the same implicit distribution.
You're just working withit in a different way.
(17:57):
I don't know if that's liketoo wonky of a question.
Uh, but yeah.
Maybe not.
I, and again, I don't knowif this answer is correct.
Is there a chance that the,
you know, there's one distribution,which is the original pre-training
corpora of these things.
I think there's potentially changes tothose distributions, maybe in what happens
(18:18):
if there's the kind of intentionalmulti-agent curation of new synthetic data
that's like weighted differently
into different distributions. Andwhich, you know, even with phenomena
like hallucinations, maybe likea downside of that could be
that, even then, impossiblethings can then be imagined and
injected back into training.
But, uh, I dunno any other,if you would agree with that?
(18:41):
Yeah, I would tend to agree.
I think, you know, we're intentionally,you know, I don't know how much
of this you could say is likealready in distribution, but
we are intentionally craftingdata, and rating that data with our
critique, and so on in order tooptimize it for a particular use case.
And so, I think that certainly doessignificantly change the behavior, whether
(19:02):
that could be achieved by some crazyamount of prompting on the original model.
Not sure, but I think it's a bit doubtful.
Awesome.
So maybe I can transition a little bit andI'll direct this next question to Alan.
So you know, Alan, yousaid you had this dream.
I think it's pretty inspiring, right?
That you've imagined something like AMIE for some time and
(19:23):
you're choosing a, I think, a hard problem, which is
directly interactingwith patients, right?
Directly, sort of interacting withpatients, trying to elicit information,
do that safely, do that robustly, andthat's a hard problem from several
perspectives, both in trying to do thisrealistically on the sort of technical
side, but then even from a safety side,also running a real study where the
(19:48):
AI is gonna interact with patients.
And what I wanna ask is,what's your approach to sort
of selecting clinical problems?
There's so many possible problemsthat you could select right from the
very first, I'd say kind of split.
You could have decided to do doctor-facingAI with all of the work from your team,
or you could have decided to sort offocus, which it seems like you are now
(20:08):
on kind of patient-facing, patient-interacting AI. And maybe you'll push
back and say that it, AMIE is, is actuallyboth, but I'm curious about how you just
think from sort of a high level first.
About what are the clinically interestingproblems for you to solve, and with
all the potential different areasthat you could apply AI from your team
at Google to how you chose kind ofAMIE in this, this set of problems.
(20:31):
And then I do afterwards, I'lljust telegraph a little bit.
I want to dig into some of thespecialist applications that you guys
are also you're highlighting recently.
Yeah, of course.
I mean, look, you know, I thinkan incredible privilege, right?
To be like a team like at Google andDeepMind like, so like getting to
work with people like Anil in kind ofa frontier lab effectively that is,
(20:53):
attempting to really solve intelligenceand use that to sort of benefit humanity.
And I think that this kind ofenvironment is very inspiring for
thinking in first principles, right?
And so, I think something I'vealways tried to do is think about
the first principles of what reallymedical intelligence and biomedical
intelligence actually comprises.
(21:14):
And hopefully, you can probably seethat in the thread of the work, like
in things like Med-PaLM, really fromvery first principles we were thinking
about to be useful in this domain,to have applications in this domain.
A very foundational question is whatknowledge is encoded in these systems?
We then went on this journey of tryingto rigorously break that down into sort
(21:35):
of testable forms that we can subjectto empirical validation and trying
to do this kind of innovation likeboldly, but also responsibly, right?
And
doing in ways where we build uponthings that like appropriately
respect the extensive expertisethat exists in this domain.
Like, I think the other amazing thingabout medicine is how universal
(21:57):
it is and how well studied it is.
And I like, I actually also believethat the people who practice
medicine, you'll find every kind oftalent in the medical profession.
And then in terms of the peoplethat affects, it's all of us.
Like we will all come into contactwith that at some point in our lives,
and that means there's this incrediblediversity of literature you can draw upon.
But when you move beyond kind of assessingthe knowledge in these systems, I think
(22:18):
then one of the things here, and you seethis both with our work on AMIE and our
work on co-scientists alike, is thesekinds of very foundational questions.
In the setting of AMIE, it's about tryingto understand how that knowledge can then
be used empathically, conversationally,and to what end, as you said, like
we've shown work in AMIE both as regardsto the use of research systems like
(22:41):
that in the hands of clinicians andpotentially also the use of systems
like that in terms of how they mightbe able to interact directly with
patients and in more than one setting.
And so, for us,
I think it's more about thefirst principles thinking.
Like, as Anil said, if you'rethen looking at medical conversation,
it's then very natural to try andunderstand how can you characterize
that? And how can you conceptualizewhat quality is in that domain?
(23:04):
And luckily there exists many years, many,many years of really rigorous thinking
about that, both from the perspective of
medical quality.
Right?
And, okay.
OSCE is being one particular construct.
I mean, I can't tell you howmany times I've sort of nervously
shuffled around a sports hall betweenpatient actors ready to show my
skills and show what I can do.
(23:25):
But, um, that's just one part of it.
It's not just the kind ofacquisition of medical information,
the right history taking skills,and the guide to uncertainty.
It's these other attributes oflike building rapport, trust, and
there's very good literature on that.
We found very good ways of actuallytrying to characterize that.
And then I think some of thecreativity is how to adapt that to
the technology setting we were in.
(23:46):
And obviously there's alot of limitations, right?
You know, particularly to
this paper we're discussing, whichwas set in the PaLM model era.
So, grounded in text,
right?
Exactly.
That's, that's, that was my question.
So, just to flush that out.
So, this is a chat bot, right?
That you're typing and interacting with?
For that first paper.
That was one of the constraintswe adopted and we did so for
(24:06):
practical reasons, right?
Like at that moment in time.
Still, maybe millions and millionsof people were interacting with
commercially available large languagemodels, I think by that time.
But they were doing so by typing andthese were at that time text-only systems.
And so, we tried to sort of adaptthese well-known frameworks like
OSCE to that technology setting.
(24:27):
And that introduces obviouslysome very important limitations.
It means that the work has tobe interpreted with a lot of
caution and hype and can also,
it can also end up doing sometimes moreharm than scientific findings do the good.
So, we also try to go into that in alot of depth in the paper and think
about that very rigorously, becauseagain, considering those things
carefully from first principles canalso be a very good way to open up
(24:51):
the most important next researchdirections that need to be looked into.
Do you see the next fewyears as having a big
focus or big emphasis on humancomputer interaction? How you present
information to patients and doctors?
Are these topics that you see yourteam studying or that, you know, you
see the field moving towards, butsomething that is not necessarily
(25:11):
your focus with AMIE?
I think if you care about AI in medicine,and if you look back at the last
decade of work in this field.
It's been a consistent thread howpeople can interact with these systems.
Like even, I mean, we were talkingearlier about medical imaging, and I
remember, you know, all this work fromconcept bottlenecks to kind of,
(25:32):
like, activation-based heat maps anddo those work or do those not work?
And you know, that lovely work actually of like other networks analyzing how
classifiers work?
Like why is a zebra a zebra?
Well, actually it turns out it's because it's activating all the
weights for being stripy and there's otheractivations going on for being horse-like.
And that may be much more usefulthan a sort of blotchy sort
of pixel-wise activation map.
(25:53):
And I think when you kind of moveinto this era that we are in now
of interactive, very powerful,conversational and multimodal systems,
but systems which can actually
sometimes go away and do work.
Like they can work for potentiallyhours, potentially even days
before coming back to you, bringingback vast swaths of information.
(26:13):
I think it's gonna be critical to optimizeand think about these things properly.
Again, look, maybe it's my bias asa doctor, but I think medicine is
the most humane of things, right?
It's the most essential human condition.
And so, our ability to actually interact with these systems properly
is gonna be very, very important.
Both for the good, right, in termsof getting the most out of them and
(26:34):
fulfilling their undoubted potential,but also to mitigate risks which could
otherwise be very significant indeed.
Alright, and maybe I can justget one last question. Alan,
I don't think I asked you this lasttime you were on, but you are, correct
me if I'm wrong here by clinicaltraining, you're a surgeon, right?
You were or you are a surgeon.
You're a practicing surgeon.
That's the one.
Yeah.
Yeah.
(26:55):
So, can I just get some last thoughtshere before we move to the lightning round
on AI's impact for surgery outside of AMIEand internal medicine and all these other
contexts that we typically discuss it.
How do you see AI changingsurgery in the next few years?
Yeah, I think it already is actually likeone way to look at this is preoperative,
intraoperative, postoperative.
There's probably like many otherways and foundational understanding
(27:16):
of the conditions themselves.
If you think about the process ofworking people up for operations,
it, depending on the specialty,
There's immense opportunity there formultimodal analysis, patient selection,
operative selection, operative technique.
The tools even that are used in theater.
These things can be revolutionized.
Like in fact, some of my researchbefore I came to DeepMind was in the
(27:38):
setting of like vascular surgery withthe idea of personalized stent grafts.
3D modeling of CT scans andthe design and durability of
like devices, right?
That can seal off an aneurysm, butyou can imagine how much more personal
and how much more precise some of thatwork is gonna become intraoperatively.
I think there's amazing potential.
There's not only potential actually forguiding operations, robotic surgery,
(28:01):
that kind of thing, but even in termsof medical education, understanding
what's going on with the procedure,quality control of the operation.
We've had systems for a whilethat can do operative phase
identification and things like that.
But the way I think that manysurgeons still train is like with
logbooks, writing down the operations.
The whole field of learning health care system, quality improvement,
(28:21):
understanding granularly, what'sgoing right, what's going wrong.
If there's a digital exhaust of sort ofrich video, biomedical data, this entire
thing can probably end up being much,much better, much, much more joyful
end-to-end, and then postoperatively.
The way in which we track supportpatients, like getting better at
home, understanding better howto do surveillance for certain
(28:43):
kinds of operative outcomes, andreintervention or in, in the setting
of like surgery feels like cancersurvivorship and things like that.
There, there's like a raft of ways it, infact, I would ask the opposite question.
Is there an aspect of surgical practice orof medical practice that we can't imagine
this incredible tool being useful for?
Terrific.
Thank you.
(29:03):
Thank you.
That was a great answer.
Alright, Andy, are we readyfor the lightning round?
We are ready for the lightning round.
So, the way that we're gonna do thisis that I'm gonna ask going to Anil,
Raj is gonna ask one to Alan and we'lljust go back and forth like that.
So, Anil, first question.
(29:24):
You mentioned in the introductionthat your brother is a role model
for you and that you look up to him.
He's also a doctor.
Uh, what kind of doctor is he?
Can you remind me?
He's
a pediatric ICU.
That's
what I thought.
So, my question for you is, if you have amedical question right now, do you go to
AMIE first or do you go to your brother?
Uh hmm.
(29:46):
I would say I probably go to mypartner who is also a doctor.
Oh, secret answer number three.
Okay.
Also very, uh, politicallysavvy answer there.
So, bonus points for that.
Alright, this one's for Alan.
Alan, if you weren't in medicine,what job would you be doing?
Oh, my goodness.
That's an incredible question.
I've sort of neverreally thought about that.
(30:07):
That is, that is a very difficultquestion to, like, it would
probably be something inscience for human benefit.
Like, I, you know, maybe it wouldbe something in, in kind of
energy or, or climate, uh, kindof science, something like that.
And it's kind of a boring answer.
I love the idea of like, using thescientific method to try and make
life a bit better for other people.
(30:28):
And so if you don't allow meto do that through biomedicine,
then I'll find some other way.
Terrific.
Great.
Uh, not, not a boring answer.
We'll, we'll take it.
Not at all.
Full disclaimer, Anil,Raj wrote this question.
Uh
oh.
No, no.
Actually, my wife, wrote thisquestion yesterday while we were
just after we put the kids down.
Yeah, go ahead, Andy.
(30:49):
Ane what was the best piece of adviceyour Ph.D. advisor ever gave you?
Uh hmm.
If you can't come up withanything, we're gonna have
this one.
This one's real, real tough.
I'm really, really strugglingto think of, uh, anything.
No.
Uh,
yeah, Mike, don't, don't editthe silence out of this one.
(31:12):
Yeah.
The best piece of advice. It's hard to pick.
I mean, I feel like, uh, there were somany, I know probably so many his advices.
I, I think, honestly just, maybe notadvice, but what you've done in terms
of pushing me to even take thisinternship and really, thinking about
what the best opportunity for growth was.
I think that mindset of looking for themost exciting opportunity and trying to
(31:36):
learn the new thing and challenge myselfis something that you've definitely
taught me and as, as a general skill.And so, I think that that is
something that I'm really thankful for.
Yeah,
thanks a lot, and thanks Rachna, for that
question.
Yeah.
Nice.
Alright, the next one is for Alan.
Will AI and medicine be driven more bycomputer scientists or by clinicians?
(31:57):
Uh, it'll be driven by both.
It'll be driven by bothtogether, and patients.
Alright, excellent.
Um, Anil, if you could havedinner with one person, dead
or alive, who would it be?
I think probably my grandpa, whopassed away when I was younger and
I think he just really would liketo see the person I grew up to be.
(32:19):
And I think he'd be
very surprised, honestly.
Great answer.
Last question on thelightning ground for Alan.
Alan, do you think things createdby artificial intelligence
can be considered art?
Uh, wow.
That that is like a, yes, but I'm very ignorant on this.
(32:41):
I'm not, I'm not a good person to ask.
We'll, we'll take it.
You guys passed the lightning round.
Well done.
Congratulations.
Yeah.
Yeah.
Alright, so for the last coupleof questions, we wanna zoom out
and ask some bigger picture ones.
I feel like Alan is still lost in thought.
Trying to think about if AI can be art.
I see he's looking. I
think, I think Alan is thinkingabout his alternative career.
(33:02):
He is outside of medicine.
Yeah.
Alright, so we're gonna askone of these to each of you.
We'll start with Anil first.
Anil, how do you see AMIE beingused in five years and when do
you think most patients' firstinteraction with the health care
system will be with a system like this?
Yeah, I mean, I think in five years,judging by kind of the pace that
(33:23):
like, I've seen things improve both,in terms of the base model as well
as the work that we're doing and manyothers are doing in this kind of space.
I feel pretty optimistic that in fiveyears we'll be at the level, like
technologically, where we would have asystem that can interact with patients
that can provide, you know, goodguidance to them and be used probably in
(33:45):
conjunction with overseeing cliniciansthat can, maybe it's a setting where this
kind of system would perform the intakeand provide a plan or something like that.
But then a clinician would be thereon the end to click a checkbox
and say like, hey, I agree with,
you know, AMIE's plan or whatever.
I will say, you know, ofcourse, like our research with
AMIE is research and I think it'san important distinction to make.
(34:07):
We're not building a product here.
But I imagine that's totally possible.
I think in five years perhaps we mighteven be at the point where it's
more safe to actually have a system thatenables access to this kind of expertise.
You know, there is like such a lack ofaccess to high quality expertise globally.
And so, I think we might be at a stagewhere it's probably even better to
(34:30):
have a semi-autonomous system likethis able to interact with patients.
And so, I don't know how this worksout, from a regulatory perspective and
like logistically, of course there's,many barriers to work through, but
I think at least like technologically,we would be at that stage
where it's totally possible.
One quick follow up.
What do you think that AMIE in 2030will have or what will it be able
(34:51):
to do that AMIE in 2025 cannot.
In 2030, I certainly think thiswill be a system that can operate
in very natural conversations.
So certainly, I don't think it'llbe like a text-based conversation.
I think that it will be able tovery seamlessly look up the most
recent guidance and you know,any kind of specialty case.
(35:12):
The advantage of AI, right,
it doesn't need to likestudy one specialty.
It can kind of have expertise ineverything and that's all, you know, kind
of complimentary, all that knowledge.
And I think the system really canperform most roles that don't require
any particular physical intervention,which it can obviously recommend.
And hopefully at that point, likesuch a system would be pretty well
(35:34):
validated in with real patients as opposedto, you know, simulated settings.
Cool.
Awesome.
Alright, this is our last question andI wanna, I wanna pose this one to Alan.
So Alan, we had you on the podcast before,and I think this was a few years ago now,
maybe a little over two years ago.
And we spoke about Med-PaLM I think youguys had pre-printed it and we were
(35:57):
maybe speaking right in between the sortof pre-print and then when the paper
eventually was published in the journal.
The question is, what do you think hasbeen the biggest change or maybe what's
been most surprising for you to see?
In the last two years since you werelast on the podcast, and then maybe also
the sort of flip, the inverse of thatquestion, what are you most surprised that
(36:18):
hasn't really changed since May of 2023?
I think on the first one, weall knew that, you know, pace was
gonna be rapid, in this field.
But I think the still even for me, the speed of improvement in so many
different ways of these base models, likeparticularly, now being in this Gemini
(36:39):
era of like natively multimodal models.That has been very surprising and is a big
shift from, some of the topics we've beentalking about today, like specializing
models, which, is much less relevant now.
The other half of the question was what,what's the same? What are you most,
yeah, yeah, exactly.
Or what are you most surprised byit, it being the same or have haven't
really changing over the last two years?
(37:01):
I think sometimes in the adventof new technology like this, it
can be quite important how theecosystem comes together around that.
I feel that it's still taking some time,
actually.
Those who develop the technologyare only one small part of this.
There's entire ecosystems who havea very big role to play in terms of.
(37:22):
How systems like this are best evaluatedhow they're best benchmarked, like what
the most important use cases are and,and how to characterize those properly.
And, you know, look, maybe it's a bitharsh to say that in two years,
not much has changed there, but I thinkthere's a lot of room for growth there.
For technology and technologists to be ledactually by the, the domains themselves.
(37:47):
Terrific.
Alright.
Thank you both for beingon AI Grand rounds.
That was, that was great.
Yeah.
Thank you both.
Yep.
Thanks for having us.
Thanks.
Thanks so much.
This copyrighted podcast from theMassachusetts Medical Society may
not be reproduced, distributed,or used for commercial purposes
without prior written permission ofthe Massachusetts Medical Society.
(38:08):
For information on reusing NEJM Grouppodcasts, please visit the permission
and licensing page at the NEJM website.
It has been a consistent thread howpeople can interact with these systems.
Like even, I mean, we were talkingearlier about medical imaging and
I remember, you know, all this workfrom concept bottlenecks to kind of,
like activation-based heat maps.
You know, that lovely workactually of other networks
analyzing how classifiers work.
Like why is a zebra, a zebra?
Well, actually it turns out it's becauseit, it's activating all the weights
(00:21):
for being stripey and there's otheractivations going on for being horse-like.
And that may be much moreuseful than a sort of blotchy,
pixel-wise activation map.
And I think when you kind of move intothis era that we are in now, systems which
can actually go away and do work, likethey can work for potentially even days
before coming back to you, bringing backvast swaths of information, I think it's
gonna be critical to sort of optimizeand think about these things properly.
(00:45):
You know, again, look, maybe it'smy bias as a doctor, but I think
medicine is the most humane
of things, right?
It's the most essential human condition,and so our ability to actually
interact with these systems properlyis gonna be very, very important.
Welcome to another episodeof NEJM AI Grand Rounds.
(01:07):
I'm your co-host Raj Manrai.
Today, Andy and I are excited tohave two guests on the podcast,
Dr. Alan Karthikesalingam of Google DeepMind,and Dr. Anil Palepu of Google.
Listeners of the podcastwill remember Alan.
He's making his second appearanceon AI Grand Rounds, and Anil
joins us for the first time.
Together they told us about Google'sarticulate medical intelligence explorer,
(01:29):
often referred to as either AMIE or AMIE.
AMIE is an AI system that's designedto directly interact with patients.
Alan and Anil took us through theproject from ideation to building
it to some of the evaluationstudies that are happening today.
It was also a great chance to reflectwith Alan on how AI has changed and
how it hasn't since he was last onthe podcast about two years ago.
(01:51):
Alan has this special ability to distillcomplex technical and clinical ideas
into their essence, and he really isa wonderful example of a physician
who's really become a leader in AI.
The NEJM AI Grand Rounds Podcastis brought to you by Microsoft,
Viz.ai, Lyric, and Elevance Health.
(02:14):
We thank them for their support.
And with that, we bring youour conversation with doctors
Alan Karthikesalingam and Anil Palepu.
Alright, well, Anil and Alan,welcome to AI Grand Rounds.
We're excited to have you today.
Yeah, thanks for having us.
Yeah.
Thanks for having us, Andy.
With the exception of Zak Kohane,
(02:35):
I think that Alan is the first repeatguest that we've had on the podcast.
So great to have you back, Alan.
I think that's right.
Yeah, that's an honor.
Way to go to
intimidate me at the start.
So as a result, Alan, we've actuallyalready asked you the following question,
so I'm gonna pose this question to Anil.
Anil, this is a question thatwe always get started with,
that we ask all of our guests.
(02:55):
Could you tell us about the trainingprocedure for your own neural network?
How did you get interested in artificialintelligence and what data and experiences
led you to where you are today?
Yeah, I did my undergrad in 2016.
That's when I started.
At the time, I think I had.
Very little idea of what I wanted to do.
I was interested in medicine,but I definitely did not
(03:18):
know that I wanted to do AI.
I don't think I knew whatAI was at that point.
I basically just knew that I didn'treally think I could be a doctor.
I was a bit too lazy.
I didn't think I could studyfor the MCAT or do all the
steps that are needed for that.
So, I was like maybe doing somethingelse like perhaps computational
would be a bit easier for me.
And that's kind of how I got started.
It's not like a glorious story of knowingI had a dream and reaching it, but as I
(03:43):
started to do research, I really fell inlove with the type of work we were doing.
And I found that I had been doing likea lot of projects alongside clinicians
and driven by a clinical need.
And I was really excited that
it felt like these people allhad some serious problems.
What really struck me was everyone wassuper unsatisfied with the status quo,
(04:04):
and there were so many things to improve.And me, as an undergrad, I actually felt
like I could apply the things I learnedin class. Like, me learning like about
random forests two weeks ago, and then
applying that to like a real problem andactually seemingly making a difference
in the workflows of these clinicianswas like crazy inspiring to me.
And that led me to be like, okay, Iwanna take this to the next level.
(04:27):
I wanna do my Ph.D. and keepdiving down in this path.
And so I did that.
I applied for my Ph.D.
I went to MIT and had the pleasureof working with an amazing advisor
there who is not paying me to saythis, but he's on this call right now.
And he taught me a lot about deeplearning and got me really excited,
particularly in the tech space for him.
That's when I joined Google.
(04:48):
Met some amazing coworkers there,Alan included, and still there today.
Keep learning from every day and reallyexcited about the mission we're pushing.
Cool before we come back to your worldclass graduate training and mentorship,
'Cause we're gonna come backand spend a lot of time on that.
I feel like this is gonnabe a recurring theme.
Yeah, it's gonna be a recurring theme.
Yeah.
So, you're yet another person fromthis HST program at Harvard and MIT.
(05:10):
So, this isn't like something thatwe've planned, but it just seems
to be that this program produces alot of folks at this intersection.
However, before you got inthis, could you take us back to
early days of Anil, when you werestill like much more pluripotent
before you had even thought aboutcomputer science and engineering.
What did you enjoy growing up andhow did you end up in a biomedical
engineering program in the first place?
As a kid, like I really looked upto, I think my family, particularly
(05:34):
my brother, I have an older brother,and he always had a very strong
vision of what he wanted to do.
And he knew he wanted to be adoctor and everyone could see it,
he really had that drive in him.
And I was like, he knows what he's doing,he's doing all the work to figure out
what's important and what to work on.
So maybe I'll do that.
And again, like I said, I think Iknew what my skillset was, I knew
(05:54):
what my limitations were, so I waslike, I don't think I could actually
do the work of seeing patientsand have that on day-to-day.
Like, I wanna sit on my couchand watch TV while I work.
That doesn't seem possible ifyou're seeing patients all day.
So, I found some path that was,I think very inspired by him.
I think I've also certainly changeda lot as well, especially as I've
(06:15):
gotten deeper into the research andI think I'm a little bit more driven,
at this point than I started out.
Awesome.
Thanks for that, Anil.
So, I think we're gonna hop in and Alanmaybe just quickly so that we don't
shortchange you, do you wanna hop back onand just quickly introduce yourself again?
Yeah, of course.
Thanks again for having me.
So, I'm Alan. I'm a clinician and aresearcher at DeepMind, a company
(06:35):
I joined kind of about eight yearsago now. And I've always worked on
AI for biomedicine, doing stintsbasically at DeepMind itself,
also in different teams in Google.
I'm very lucky to work withAnil for a good chunk of that
on some very exciting work.
Awesome, thanks.
So now we're gonna transition andtalk about some of the projects that
you both have worked on together.
And I'll preface this by saying the firstpaper that we're gonna discuss, again,
(07:00):
Zak Kohane mentioned this in our scotchinfused end-of-the-year round table.
Did you
guys, wait just as a, didyou guys listen to that one?
So it's a good one.
We discussed AMIE, quite a bit,so I think, you'll enjoy that.
But anyway, go ahead.
Yeah.
Yeah.
And one of the things we discussed isactually the correct pronunciation.
Is it AMIE or is it AMIE?
I think this is, uh, is it potato, or is it?
(07:25):
Yeah, ask three doctors.
You'll get three answers.
One of the things about a transatlanticteam is I think we find, like, almost
every way to pronounce almost every word.
So, we'll leave that one a mystery for you.
Okay.
Well, uh, in my slightly southerntwang, I'm gonna say AMIE here
forward and Raj can go with the morecosmopolitan pronunciation of AMIE.
Okay, sounds
(07:46):
good.
Um, so anyway, we had this end ofthe year episode with Zak, and one
of the things that we do is round uppapers that have been published at
NEJM AI, and then we talk about papersthat have been published elsewhere.
And Zak and I actually both pointedto the paper that we're gonna talk
about next as one of the papersthat we thought was most impactful,
that was not published at NEJM AI.
(08:07):
And so, the paper is "Towards conversationaldiagnostic artificial intelligence."
And instead of doing you both thedisservice of butchering the explanation
set up and context how about we goto Anil and you give us the technical
overview and the technical setup, andthen we'll go back to Alan and discuss
some of the clinical aspects of it.
Yeah.
So, this paper "Towards conversationaldiagnostic AI," essentially what we're
(08:28):
dealing with is up until this point, therewas a lot of work showing essentially
that, these LLM based systems do exhibitsome quality medical reasoning. They were
tested on these benchmarks like MedQA,and, you know, at least in like these
multiple-choice settings, they performedquite well and often superhuman.
That being said, there was verylittle work investigating how well
(08:51):
these systems can actually elicitthis information from patients.
And so, this is a setting where ratherthan having the entire picture provided
upfront and an LLM simply interpreting it.
In a real clinic, a clinician willhave to actually talk to a patient, get
the information from them, do so in away where they're exhibiting empathy
(09:12):
and quality conversational skillsfor history taking, and ultimately
still come up with the correct planand differential for the patient.
So, this was what we really wanted to test.
Alan and others really set a really greatvision for how we would test this, which
is through something called an OSCE orObjective Structured Clinical Examination.
And so, we modeled our study aroundtext-based OSCE and we tested whether
(09:37):
our system that we developed couldperform this kind of consultation
with patient actors at the levelof primary care physicians.
And so that's what we did for the study.
In terms of the actual modeling,at the time we were working with
PaLM-based models and reallythe key innovation here was
we were trying to simulate conversationsbetween doctors and patients and use that
(10:01):
to actually train a model to emulate therole of a doctor in those conversations.
Can I hop in here?
'cause I remember you actuallypresented this at a lab meeting one
time, and there was a couple technicalaspects that I thought were just,
like, both surprising and interesting.
If I remember correctly, you have this
team of PaLM models, one is acting asthe doctor, one's acting as the patient.
(10:24):
And you use this ensemble of LargeLanguage Models to synthetically generate
data, and then you train on that dataand the model keeps getting better.
And I remember at the time I was deeplyskeptical of pulling yourself up by
your own bootstraps aspect of whatpeople are talking about in synthetic
data, but this seemed to be a prettycompelling example of where you could
have the model generate syntheticdata and it get better at the task
(10:45):
that you're trying to get it to learn.
So, maybe could you talk a littlebit about, like, that aspect?
Was that surprising to you or have youseen other examples of this sort of
synthetic data trick working in practice?
Yeah, so our setup, as you describedreally well, is we had this multi-agent
system where we had essentially wehad that doctor and that patient each
played by PaLM at the time and theywere interacting with each other.
(11:07):
And we also had a separate agent, amoderator, which was just basically
checking whether conversation was done.
And really the criticalagent was this critic agent,
which, after an entire conversationwas complete, it would review the
conversation, identify things thatthe doctor did right or wrong,
and pass that to the doctor.
The doctor got to trythe conversation again.
And in doing so, we were able to guidethe conversation to words, what was better
(11:32):
doctor behavior according to how we knewwe'd be evaluated and what the optimal
plan for this patient would've been.
The other aspect of this that I thinkis injecting new knowledge is the fact
that our conditions, our vignettes,were actually conditioned on search.
And so essentially here we're ableto bring in some kind of grounding
(11:54):
that is able to be providedto the model during training.
I think the other important point tonote is that these PaLM-based models
initially just like simply could not,if you tried to prompt them to do this,
they would not hold a conversation.
Certainly not over this many turns.
And so, I think a lot of our gainswere really in being able to hold
(12:14):
that conversation, like the style of
that diagnostic dialogue.
Whereas like today, we don't actually needto always do significant post training.
Like in our most recent work, we'reactually using the base model for
most of the heavy lifting there.
So I think that is somethingthat has also just changed over
time in terms of necessity.
So cool.
(12:35):
A couple other things now that thisis all loaded into my context window.
I remember the really surprisingpart of this is that you had all
of this presumably expensive humanOSCE data of actual conversations.
And one of the conclusions I rememberreading in the paper was that actually
the synthetic data ended up beingmore valuable than the human data.
Is that a correct recollection?
(12:56):
I guess valuable in what sense?
In that just training on thehuman data was insufficient.
And you really needed the synthetic datato be able to get the model to learn.
Oh yeah.
So, we had a human, like we hadtranscripts, which we were training
on, and I think, yeah, there were manylimitations with those transcripts.
I'm sure if we have, like,
(13:16):
human data in the exact style ofthe kind of consultations we wanted,
that would be a different story.
But what we're dealing withis very noisy transcripts that
maybe didn't cover the range ofconditions we wanted to cover.
And so, I think in that sense,by synthetically creating them,
we're able to actually tailorwhat we're providing to the model.
But I think a large part of thatwas A, like that it was an audio
(13:39):
transcript and B, that we were able toflexibly choose the set of conditions.
Yeah, to add a bit of color to this, youknow, you can imagine, I suppose it's
like one thing for imitation learningto have a worked example of exactly how
you would want an agent to behave itself.
It's quite another thing to be doingSFT on some transcripts, which are
(13:59):
literally just transcribed from
video consultations in which the providerand the patient might exchange a joke,
but about something that has visuallyjust sort of happened, like a bird
flew outside the window or something.
And there's a lot of what I just did,you know, a lot of, um, ahhh, but, you
know, this kind of thing faithfullytranscribed so that, plus I think Anil
(14:19):
makes a great point about the breadthand depth of conditions, and one of the
many beautiful things about medicineis that it is essentially a long tail.
You know, even if you take commonconditions, there's a kind of universe
of ways in which even the most commonor garden thing can actually present
differently in different people.
And so, the ability of utilizing self-play,I think one wonderful thing about that is
(14:42):
the ability to permute those situationsto bring search into the loop, which
means you're not relying on memory that'son information that like happened to be
encoded in at pre-training and to muchmore fully go beyond like the experience,
right, of what happens to have beenwritten down in any one data set.
Alan, that sort of reminds meof with image-based models,
the augmentations, right?
(15:04):
The sort of flips, and the permutations, andthe rotations, and blurring, and shearing,
other things that we do that are nowjust kind of standard practice, right?
You're sort of permuting all the ways topresent information to the model and then
trying to see what emerges out of itthat's more stable and more robust.
Yes.
Is that, is that fair?
Yeah.
I love that analogy.
So, you know, there was, there's been thiskind of arc over the last eight years in
(15:25):
which I think for, you know, I've alwayshad this kind of dream of systems like
AMIE, and it was one of the reasons whylike baffling my family, I left a very
lovely and amazing academicclinical practice, right?
And, like, a very nice lab andeverything to come to DeepMind.
And you know, there has been this arc fromkind of supervised learning and imitation
learning of these point tasks, producingthese systems that were useful for one
(15:47):
thing but fragile and not generalizable.
And not robust.
And in the era just precedingthe advent of these amazing LLMs,
we were doing a lot of work.
Trying to address those limitationsthrough techniques like their self
supervised learning especially, and weactually had a paper in Nature Medicine
on the use of synthetic data in theradiology context and the dermatology
(16:11):
context specifically for doing exactlywhat you were saying, but imagining,
you know, those augmentation regimes,but actually grounded in the domain.
And even there again, we found somepretty great impacts, including
non-trivial improvements to things likefairness in that context, and again,
I think for medicine, this becomesimportant because of the long tail.
(16:32):
So, when you think about all the ways inwhich a condition can present and all the
intersectional groups you actually haveto account for, it feels to me like an
almost impossible challenge to try and getthat distribution represented properly.
Under like a supervised learningregime or under like a regime of real
data that is going to be difficult.
(16:53):
It's kind of like a car that hasto learn how to drive by memorizing
every possible road situation.
It just doesn't seem feasible.
And so being able to leverage selfplay, synthetic data regimes, the
intelligent acquisition or likemodification of those data, like
AI, that feels like a very powerfulparadigm for medicine and life sciences.
Can I ask a technicalquestion on that, Alan?
(17:13):
And feel free to
take a pass on this.
So, where does the information come from?
So, you have this long tail of conditions,you're pulling data out of the weights
of the model, and so presumably,like, no new information is created.
I understand that, like, supervised finetuning post-training argument where now
you're getting the model to be in theright shape of things, meaning that it's
(17:34):
having this like turn-by-turn dialogue.
But what's your senseof this sort of, like,
added the information gain from doingthis because in, in self-play, like
Go, the model can actually play gamesthat have never been played before
because you have this perfect simulatorof Go, but in the case of generating
synthetic data, you're drawing fromsome the same implicit distribution.
You're just working withit in a different way.
(17:57):
I don't know if that's liketoo wonky of a question.
Uh, but yeah.
Maybe not.
I, and again, I don't knowif this answer is correct.
Is there a chance that the,
you know, there's one distribution,which is the original pre-training
corpora of these things.
I think there's potentially changes tothose distributions, maybe in what happens
(18:18):
if there's the kind of intentionalmulti-agent curation of new synthetic data
that's like weighted differently
into different distributions. Andwhich, you know, even with phenomena
like hallucinations, maybe likea downside of that could be
that, even then, impossiblethings can then be imagined and
injected back into training.
But, uh, I dunno any other,if you would agree with that?
(18:41):
Yeah, I would tend to agree.
I think, you know, we're intentionally,you know, I don't know how much
of this you could say is likealready in distribution, but
we are intentionally craftingdata, and rating that data with our
critique, and so on in order tooptimize it for a particular use case.
And so, I think that certainly doessignificantly change the behavior, whether
(19:02):
that could be achieved by some crazyamount of prompting on the original model.
Not sure, but I think it's a bit doubtful.
Awesome.
So maybe I can transition a little bit andI'll direct this next question to Alan.
So you know, Alan, yousaid you had this dream.
I think it's pretty inspiring, right?
That you've imagined something like AMIE for some time and
(19:23):
you're choosing a, I think, a hard problem, which is
directly interactingwith patients, right?
Directly, sort of interacting withpatients, trying to elicit information,
do that safely, do that robustly, andthat's a hard problem from several
perspectives, both in trying to do thisrealistically on the sort of technical
side, but then even from a safety side,also running a real study where the
(19:48):
AI is gonna interact with patients.
And what I wanna ask is,what's your approach to sort
of selecting clinical problems?
There's so many possible problemsthat you could select right from the
very first, I'd say kind of split.
You could have decided to do doctor-facingAI with all of the work from your team,
or you could have decided to sort offocus, which it seems like you are now
(20:08):
on kind of patient-facing, patient-interacting AI. And maybe you'll push
back and say that it, AMIE is, is actuallyboth, but I'm curious about how you just
think from sort of a high level first.
About what are the clinically interestingproblems for you to solve, and with
all the potential different areasthat you could apply AI from your team
at Google to how you chose kind ofAMIE in this, this set of problems.
(20:31):
And then I do afterwards, I'lljust telegraph a little bit.
I want to dig into some of thespecialist applications that you guys
are also you're highlighting recently.
Yeah, of course.
I mean, look, you know, I thinkan incredible privilege, right?
To be like a team like at Google andDeepMind like, so like getting to
work with people like Anil in kind ofa frontier lab effectively that is,
(20:53):
attempting to really solve intelligenceand use that to sort of benefit humanity.
And I think that this kind ofenvironment is very inspiring for
thinking in first principles, right?
And so, I think something I'vealways tried to do is think about
the first principles of what reallymedical intelligence and biomedical
intelligence actually comprises.
(21:14):
And hopefully, you can probably seethat in the thread of the work, like
in things like Med-PaLM, really fromvery first principles we were thinking
about to be useful in this domain,to have applications in this domain.
A very foundational question is whatknowledge is encoded in these systems?
We then went on this journey of tryingto rigorously break that down into sort
(21:35):
of testable forms that we can subjectto empirical validation and trying
to do this kind of innovation likeboldly, but also responsibly, right?
And
doing in ways where we build uponthings that like appropriately
respect the extensive expertisethat exists in this domain.
Like, I think the other amazing thingabout medicine is how universal
(21:57):
it is and how well studied it is.
And I like, I actually also believethat the people who practice
medicine, you'll find every kind oftalent in the medical profession.
And then in terms of the peoplethat affects, it's all of us.
Like we will all come into contactwith that at some point in our lives,
and that means there's this incrediblediversity of literature you can draw upon.
But when you move beyond kind of assessingthe knowledge in these systems, I think
(22:18):
then one of the things here, and you seethis both with our work on AMIE and our
work on co-scientists alike, is thesekinds of very foundational questions.
In the setting of AMIE, it's about tryingto understand how that knowledge can then
be used empathically, conversationally,and to what end, as you said, like
we've shown work in AMIE both as regardsto the use of research systems like
(22:41):
that in the hands of clinicians andpotentially also the use of systems
like that in terms of how they mightbe able to interact directly with
patients and in more than one setting.
And so, for us,
I think it's more about thefirst principles thinking.
Like, as Anil said, if you'rethen looking at medical conversation,
it's then very natural to try andunderstand how can you characterize
that? And how can you conceptualizewhat quality is in that domain?
(23:04):
And luckily there exists many years, many,many years of really rigorous thinking
about that, both from the perspective of
medical quality.
Right?
And, okay.
OSCE is being one particular construct.
I mean, I can't tell you howmany times I've sort of nervously
shuffled around a sports hall betweenpatient actors ready to show my
skills and show what I can do.
(23:25):
But, um, that's just one part of it.
It's not just the kind ofacquisition of medical information,
the right history taking skills,and the guide to uncertainty.
It's these other attributes oflike building rapport, trust, and
there's very good literature on that.
We found very good ways of actuallytrying to characterize that.
And then I think some of thecreativity is how to adapt that to
the technology setting we were in.
(23:46):
And obviously there's alot of limitations, right?
You know, particularly to
this paper we're discussing, whichwas set in the PaLM model era.
So, grounded in text,
right?
Exactly.
That's, that's, that was my question.
So, just to flush that out.
So, this is a chat bot, right?
That you're typing and interacting with?
For that first paper.
That was one of the constraintswe adopted and we did so for
(24:06):
practical reasons, right?
Like at that moment in time.
Still, maybe millions and millionsof people were interacting with
commercially available large languagemodels, I think by that time.
But they were doing so by typing andthese were at that time text-only systems.
And so, we tried to sort of adaptthese well-known frameworks like
OSCE to that technology setting.
(24:27):
And that introduces obviouslysome very important limitations.
It means that the work has tobe interpreted with a lot of
caution and hype and can also,
it can also end up doing sometimes moreharm than scientific findings do the good.
So, we also try to go into that in alot of depth in the paper and think
about that very rigorously, becauseagain, considering those things
carefully from first principles canalso be a very good way to open up
(24:51):
the most important next researchdirections that need to be looked into.
Do you see the next fewyears as having a big
focus or big emphasis on humancomputer interaction? How you present
information to patients and doctors?
Are these topics that you see yourteam studying or that, you know, you
see the field moving towards, butsomething that is not necessarily
(25:11):
your focus with AMIE?
I think if you care about AI in medicine,and if you look back at the last
decade of work in this field.
It's been a consistent thread howpeople can interact with these systems.
Like even, I mean, we were talkingearlier about medical imaging, and I
remember, you know, all this work fromconcept bottlenecks to kind of,
(25:32):
like, activation-based heat maps anddo those work or do those not work?
And you know, that lovely work actually of like other networks analyzing how
classifiers work?
Like why is a zebra a zebra?
Well, actually it turns out it's because it's activating all the
weights for being stripy and there's otheractivations going on for being horse-like.
And that may be much more usefulthan a sort of blotchy sort
of pixel-wise activation map.
(25:53):
And I think when you kind of moveinto this era that we are in now
of interactive, very powerful,conversational and multimodal systems,
but systems which can actually
sometimes go away and do work.
Like they can work for potentiallyhours, potentially even days
before coming back to you, bringingback vast swaths of information.
(26:13):
I think it's gonna be critical to optimizeand think about these things properly.
Again, look, maybe it's my bias asa doctor, but I think medicine is
the most humane of things, right?
It's the most essential human condition.
And so, our ability to actually interact with these systems properly
is gonna be very, very important.
Both for the good, right, in termsof getting the most out of them and
(26:34):
fulfilling their undoubted potential,but also to mitigate risks which could
otherwise be very significant indeed.
Alright, and maybe I can justget one last question. Alan,
I don't think I asked you this lasttime you were on, but you are, correct
me if I'm wrong here by clinicaltraining, you're a surgeon, right?
You were or you are a surgeon.
You're a practicing surgeon.
That's the one.
Yeah.
Yeah.
(26:55):
So, can I just get some last thoughtshere before we move to the lightning round
on AI's impact for surgery outside of AMIEand internal medicine and all these other
contexts that we typically discuss it.
How do you see AI changingsurgery in the next few years?
Yeah, I think it already is actually likeone way to look at this is preoperative,
intraoperative, postoperative.
There's probably like many otherways and foundational understanding
(27:16):
of the conditions themselves.
If you think about the process ofworking people up for operations,
it, depending on the specialty,
There's immense opportunity there formultimodal analysis, patient selection,
operative selection, operative technique.
The tools even that are used in theater.
These things can be revolutionized.
Like in fact, some of my researchbefore I came to DeepMind was in the
(27:38):
setting of like vascular surgery withthe idea of personalized stent grafts.
3D modeling of CT scans andthe design and durability of
like devices, right?
That can seal off an aneurysm, butyou can imagine how much more personal
and how much more precise some of thatwork is gonna become intraoperatively.
I think there's amazing potential.
There's not only potential actually forguiding operations, robotic surgery,
(28:01):
that kind of thing, but even in termsof medical education, understanding
what's going on with the procedure,quality control of the operation.
We've had systems for a whilethat can do operative phase
identification and things like that.
But the way I think that manysurgeons still train is like with
logbooks, writing down the operations.
The whole field of learning health care system, quality improvement,
(28:21):
understanding granularly, what'sgoing right, what's going wrong.
If there's a digital exhaust of sort ofrich video, biomedical data, this entire
thing can probably end up being much,much better, much, much more joyful
end-to-end, and then postoperatively.
The way in which we track supportpatients, like getting better at
home, understanding better howto do surveillance for certain
(28:43):
kinds of operative outcomes, andreintervention or in, in the setting
of like surgery feels like cancersurvivorship and things like that.
There, there's like a raft of ways it, infact, I would ask the opposite question.
Is there an aspect of surgical practice orof medical practice that we can't imagine
this incredible tool being useful for?
Terrific.
Thank you.
(29:03):
Thank you.
That was a great answer.
Alright, Andy, are we readyfor the lightning round?
We are ready for the lightning round.
So, the way that we're gonna do thisis that I'm gonna ask going to Anil,
Raj is gonna ask one to Alan and we'lljust go back and forth like that.
So, Anil, first question.
(29:24):
You mentioned in the introductionthat your brother is a role model
for you and that you look up to him.
He's also a doctor.
Uh, what kind of doctor is he?
Can you remind me?
He's
a pediatric ICU.
That's
what I thought.
So, my question for you is, if you have amedical question right now, do you go to
AMIE first or do you go to your brother?
Uh hmm.
(29:46):
I would say I probably go to mypartner who is also a doctor.
Oh, secret answer number three.
Okay.
Also very, uh, politicallysavvy answer there.
So, bonus points for that.
Alright, this one's for Alan.
Alan, if you weren't in medicine,what job would you be doing?
Oh, my goodness.
That's an incredible question.
I've sort of neverreally thought about that.
(30:07):
That is, that is a very difficultquestion to, like, it would
probably be something inscience for human benefit.
Like, I, you know, maybe it wouldbe something in, in kind of
energy or, or climate, uh, kindof science, something like that.
And it's kind of a boring answer.
I love the idea of like, using thescientific method to try and make
life a bit better for other people.
(30:28):
And so if you don't allow meto do that through biomedicine,
then I'll find some other way.
Terrific.
Great.
Uh, not, not a boring answer.
We'll, we'll take it.
Not at all.
Full disclaimer, Anil,Raj wrote this question.
Uh
oh.
No, no.
Actually, my wife, wrote thisquestion yesterday while we were
just after we put the kids down.
Yeah, go ahead, Andy.
(30:49):
Ane what was the best piece of adviceyour Ph.D. advisor ever gave you?
Uh hmm.
If you can't come up withanything, we're gonna have
this one.
This one's real, real tough.
I'm really, really strugglingto think of, uh, anything.
No.
Uh,
yeah, Mike, don't, don't editthe silence out of this one.
(31:12):
Yeah.
The best piece of advice. It's hard to pick.
I mean, I feel like, uh, there were somany, I know probably so many his advices.
I, I think, honestly just, maybe notadvice, but what you've done in terms
of pushing me to even take thisinternship and really, thinking about
what the best opportunity for growth was.
I think that mindset of looking for themost exciting opportunity and trying to
(31:36):
learn the new thing and challenge myselfis something that you've definitely
taught me and as, as a general skill.And so, I think that that is
something that I'm really thankful for.
Yeah,
thanks a lot, and thanks Rachna, for that
question.
Yeah.
Nice.
Alright, the next one is for Alan.
Will AI and medicine be driven more bycomputer scientists or by clinicians?
(31:57):
Uh, it'll be driven by both.
It'll be driven by bothtogether, and patients.
Alright, excellent.
Um, Anil, if you could havedinner with one person, dead
or alive, who would it be?
I think probably my grandpa, whopassed away when I was younger and
I think he just really would liketo see the person I grew up to be.
(32:19):
And I think he'd be
very surprised, honestly.
Great answer.
Last question on thelightning ground for Alan.
Alan, do you think things createdby artificial intelligence
can be considered art?
Uh, wow.
That that is like a, yes, but I'm very ignorant on this.
(32:41):
I'm not, I'm not a good person to ask.
We'll, we'll take it.
You guys passed the lightning round.
Well done.
Congratulations.
Yeah.
Yeah.
Alright, so for the last coupleof questions, we wanna zoom out
and ask some bigger picture ones.
I feel like Alan is still lost in thought.
Trying to think about if AI can be art.
I see he's looking. I
think, I think Alan is thinkingabout his alternative career.
(33:02):
He is outside of medicine.
Yeah.
Alright, so we're gonna askone of these to each of you.
We'll start with Anil first.
Anil, how do you see AMIE beingused in five years and when do
you think most patients' firstinteraction with the health care
system will be with a system like this?
Yeah, I mean, I think in five years,judging by kind of the pace that
(33:23):
like, I've seen things improve both,in terms of the base model as well
as the work that we're doing and manyothers are doing in this kind of space.
I feel pretty optimistic that in fiveyears we'll be at the level, like
technologically, where we would have asystem that can interact with patients
that can provide, you know, goodguidance to them and be used probably in
(33:45):
conjunction with overseeing cliniciansthat can, maybe it's a setting where this
kind of system would perform the intakeand provide a plan or something like that.
But then a clinician would be thereon the end to click a checkbox
and say like, hey, I agree with,
you know, AMIE's plan or whatever.
I will say, you know, ofcourse, like our research with
AMIE is research and I think it'san important distinction to make.
(34:07):
We're not building a product here.
But I imagine that's totally possible.
I think in five years perhaps we mighteven be at the point where it's
more safe to actually have a system thatenables access to this kind of expertise.
You know, there is like such a lack ofaccess to high quality expertise globally.
And so, I think we might be at a stagewhere it's probably even better to
(34:30):
have a semi-autonomous system likethis able to interact with patients.
And so, I don't know how this worksout, from a regulatory perspective and
like logistically, of course there's,many barriers to work through, but
I think at least like technologically,we would be at that stage
where it's totally possible.
One quick follow up.
What do you think that AMIE in 2030will have or what will it be able
(34:51):
to do that AMIE in 2025 cannot.
In 2030, I certainly think thiswill be a system that can operate
in very natural conversations.
So certainly, I don't think it'llbe like a text-based conversation.
I think that it will be able tovery seamlessly look up the most
recent guidance and you know,any kind of specialty case.
(35:12):
The advantage of AI, right,
it doesn't need to likestudy one specialty.
It can kind of have expertise ineverything and that's all, you know, kind
of complimentary, all that knowledge.
And I think the system really canperform most roles that don't require
any particular physical intervention,which it can obviously recommend.
And hopefully at that point, likesuch a system would be pretty well
(35:34):
validated in with real patients as opposedto, you know, simulated settings.
Cool.
Awesome.
Alright, this is our last question andI wanna, I wanna pose this one to Alan.
So Alan, we had you on the podcast before,and I think this was a few years ago now,
maybe a little over two years ago.
And we spoke about Med-PaLM I think youguys had pre-printed it and we were
(35:57):
maybe speaking right in between the sortof pre-print and then when the paper
eventually was published in the journal.
The question is, what do you think hasbeen the biggest change or maybe what's
been most surprising for you to see?
In the last two years since you werelast on the podcast, and then maybe also
the sort of flip, the inverse of thatquestion, what are you most surprised that
(36:18):
hasn't really changed since May of 2023?
I think on the first one, weall knew that, you know, pace was
gonna be rapid, in this field.
But I think the still even for me, the speed of improvement in so many
different ways of these base models, likeparticularly, now being in this Gemini
(36:39):
era of like natively multimodal models.That has been very surprising and is a big
shift from, some of the topics we've beentalking about today, like specializing
models, which, is much less relevant now.
The other half of the question was what,what's the same? What are you most,
yeah, yeah, exactly.
Or what are you most surprised byit, it being the same or have haven't
really changing over the last two years?
(37:01):
I think sometimes in the adventof new technology like this, it
can be quite important how theecosystem comes together around that.
I feel that it's still taking some time,
actually.
Those who develop the technologyare only one small part of this.
There's entire ecosystems who havea very big role to play in terms of.
(37:22):
How systems like this are best evaluatedhow they're best benchmarked, like what
the most important use cases are and,and how to characterize those properly.
And, you know, look, maybe it's a bitharsh to say that in two years,
not much has changed there, but I thinkthere's a lot of room for growth there.
For technology and technologists to be ledactually by the, the domains themselves.
(37:47):
Terrific.
Alright.
Thank you both for beingon AI Grand rounds.
That was, that was great.
Yeah.
Thank you both.
Yep.
Thanks for having us.
Thanks.
Thanks so much.
This copyrighted podcast from theMassachusetts Medical Society may
not be reproduced, distributed,or used for commercial purposes
without prior written permission ofthe Massachusetts Medical Society.
(38:08):
For information on reusing NEJM Grouppodcasts, please visit the permission
and licensing page at the NEJM website.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.