What our digital lives reveal about health with Yulin Hswen

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:05):
Welcome to Complicating the Narrative, a podcast that takes on the challenging questionsof public health by embracing complexity rather than avoiding it.
I'm your host, Salma Abdalla We often think about health environments as physical spaces,the neighborhoods we live in, the air we breathe, and the food available in our
communities.
But increasingly, our environments are also digital.

(00:25):
Social platforms, search engines, and now AI systems have shaped how we learn abouthealth,
what we fear and what we believe is possible.
Yet we often treat these technologies as tools for communication rather than environmentsthat shape population health in their own right.
These technologies have the potential to unlock a new era in clinical public healthresearch and practice, but they also have many limitations that may exacerbate existing

(00:50):
health disparities.
I believe we're past the point where digital footprints, machine learning and AI can beseen as niche
areas of research interest within public health and medicine.
I think we need to have more conversations about how to incorporate them in our work,where the blind spots are, and who's being left out as we increasingly rely on them.

(01:12):
My guest today has been thinking about the best way to use these technologies in aresponsible way.
Dr.
Yulin Hswen is an associate professor at UCSF whose work examines how online behavior andalgorithmic systems influence population health,
and how we can use big data and AI responsibly to understand society's health patterns.

(01:33):
She also serves as an associate editor at JAMA and JAMA + AI, giving her a unique vantagepoint on how the field is evolving and what standards are needed to maintain scientific
rigor.
Yulin welcome to the podcast.
Thank you so much for having me, Salma.
Yeah, so I usually try to start with a journey because I think a lot of people really tryto connect what got you interested in a certain field and ultimately where you ended up

(01:58):
focusing on with your research.
And I can see from your journey that you have a lot of experiences in epidemiology, publichealth, but also in data science.
I'm not going to mention acting for now.
We'll leave that for later.
But I wanted to first know what drew you to thinking about health from a digital and acomputational science lens.

(02:19):
So it's interesting because no path is ever straight.
So I went to school at the Harvard School of TH Chan School Public Health.
And I was very classically trained in epidemiology and biostatistics.
um And so I had that base of ultimately really trying to shift populations to behealthier,

(02:48):
right?
And then also from a lens of disparity.
So not increasing the gap, but trying to reduce it.
uh But what actually got me interested in the field of data science, but really onlinedata, is really from personal experience.
I think a lot of

(03:09):
I think passion in general for people comes from an area where you've had that type ofexperience that really fundamentally changes how you think or how you act.
And one of those was, you know, going through my doctoral program, I got quite ill.

(03:30):
and I went to go see a physician and I had a really terrible experience.
They were really dismissive, um just quite disparaging over my health uh condition.
My eyes were extremely swollen.
I really couldn't see out of them and I felt very gaslit that

(03:51):
it was, you know, it's all in my head, even though it it was visibly there, but theydismissed it as it was nothing.
And so, you know, for me, it made me go to the internet and made me go online to searchfor what was wrong and to figure out my health problem because I didn't want to go back.
I didn't want to go back to go to the health system, which I have health insurance, buttook me forever to see a doctor in Boston, Massachusetts, one of the supposed best

(04:22):
healthcare systems in the United States.
And going online, I really realized that so many other people were feeling the same way.
They put so much health information online, but that...
we were seeking so much information online.
Whether or not, let's remove beyond the fact that whether it's accurate, whether there isissues with, um again, uh knowing whether or not it's evidence-based, but the fact that I

(04:57):
did not wanna go see, go to the traditional health route and I was in the field,
really made me think that, this is where the field is going to, and this is where peopleare turning to, and we need to listen to them.
And we also can really actually better understand people's health conditions, but we canalso potentially intervene as well.
So that's really where I really started.

(05:19):
And then going forward, trying to find out who is doing that type of research.
And it ended up being Professor John Brownstein at the computational.
health epidemiology lab uh at Harvard Medical School.
And so doing research in that lab where they were collecting this type of online data andbeing able to predict disease.

(05:41):
I mean, it was epidemics at the time, now it's pandemics.
But in general, I found that you could actually use this data to be able to model what washappening.
And then I kind of took it a step forward and coming from the background again, from that
kind of disparities lens, but also, you know, looking at social factors, socialdeterminants of health, took it to more along the lines of kind of understanding social

(06:08):
norms and behaviors versus like infectious disease.
Cause I became very interested in understanding how those type of, you know, shifts inpeople's just overall beliefs and behaviors was really being seen online first before.
you would actually see it kind of in the clinics again, because of all those factors thatI spoke about and much more.

(06:32):
um I think that's actually really interesting because we...
um
in Sudan and again, not a place that you would think of as so I'm from Sudan, my family'sin Sudan, not a place that you think of that would have a lot of technological advancement
or even a lot of data you can trace digitally.
It was very interesting to see that there are a lot of things that I hear from my motherbefore I go to the clinic that she saw through WhatsApp messages that then I actually get

(07:00):
my patients talking to me about them.
And those are not things that I can hear when talking to my colleagues or even things thatI was
studying in medical school.
It was just things that my mom actually would talk about them.
And then I will see that in the real world.
But I never thought there is a good way to do that with WhatsApp, just because we knowWhatsApp has a weird encryption that wouldn't allow us actually to track this information.

(07:21):
But for a while, my mom became some sort of a source for me.
If I'm in that primary care setting, what to expect if people in the neighborhood arecoming?
What type of information they might actually be talking about that they're not gettingfrom us, they're getting from each other or other digital traces.
I actually do think, that's a really untapped areas.
uh And I always think about it from the lens of someone in a country where my mom mightnot necessarily have access to really helpful YouTube videos that she can listen to, but

(07:45):
can just have immediate access to WhatsApp messages that are translated.
That could be accurate or not, but certainly they are going to be reflected in her healthbehavior.
So that got me really interested in the type of work that you're doing.
So if I looked up your name online, it would say you're a computational epidemiologist.
Can you tell our
listeners what that means, people who might not be familiar with the term, and how is thatdifferent from someone like me who would call myself a traditional epidemiologist.

(08:10):
I think it's really the computational part really is kind of the blending of computerscience methods and then using traditional epidemiological kind of study designs.
I would say that that's probably the best way to describe it.
The component that I add is the kind of big data, but the online data.

(08:33):
So a lot of the times, again, the data that I'm using is in the millions.
Right?
And you don't necessarily have that in the, in always the traditional form, right, and soyou need to use all these other types of kind of computational methods to be able to
aggregate the data, to collect it, to analyze it, and then again these just kind ofdifferent layers of, complexity that,

(09:00):
is, I think, different, again, just different types of methods that are used in thecomputer science field that is not always used in the traditional epidemiology realm,
right?
So for instance, like natural language processing an area of interest of mine isunderstanding human emotions and sentiment and how that spreads throughout populations,
because I think that, again, we are not logical creatures, but we're emotional and we makedecisions based on emotion.

(09:26):
So if you can track that,
and you have to find a way to kind of analyze that um is a method that isn't really used,I don't think, as much in the traditional epidemiological sense.
um Yes, that's what that an example.
the hope then, and not hope, maybe even the future would be given that we have a lot moreaccess to big data, that one day every person who's quoted as a traditional epidemiologist

(09:56):
is someone who's also using computational science?
Or do you think there's still space for people who are not necessarily using computationalscience methods?
Yeah, I mean, I don't think traditional, I'm actually a very big traditional person ingeneral.
I think that tradition needs to remain.
So like that's all the way from like, you know, traditional all the way from, qualitativemethods to survival analysis, right?

(10:21):
I don't think that those ever cannot be taught.
I think those are very important.
I think, there's, there's
There's different categories, right?
You can either have someone that has training and kind of both, that does both, or whichis actually more of my hope for the future is more interdisciplinary where you

(10:42):
have someone that is very traditionally trained as an expert and is still growing thatfield, the traditional methods like causal, like inference, you know, methods, right, that
are still very traditional epidemiology.
And then you have, you know, computer science methods that, that
are an expert in that area, right?

(11:03):
From generative AI, right?
To multimodal kind of analysis.
And they work together to be able to kind of solve these world problems, right?
Because I think it is, I mean, it is a challenge in general.
especially just in the fields, overall, I think in this world, things are moving so fast,right?
But it's hard to keep, you know...

(11:25):
It's like reading different textbooks or a different book, right?
It's hard to be so updated, in every single field.
But if you are an expert in a single field, you can kind of keep moving on in that area.
And, you know, And that's also how, traditional kind of grants work, right?
You have an expert that's in...
ophthalmology or neurology.
And you have someone that's an expert in epidemiology or again, communication, forinstance, right?

(11:50):
I'm not, I'm not a communication expert, but you have to work with people in communicationbecause goodness knows that, you know, as epidemiologists are terrible at communicating,
which is, which is, mean, I think, you know, if not 80%, 90 % of the battle, I think mostof the time that we're not very good at.

(12:11):
Yeah, no, no, I fully agree with that, especially the part about us being
not so great at communication as epidemiologists.
But so we use the term big data.
I think so I've been going through your research, as I say to all this with people here, Iswear I'm not stalking you.
But there are a few really good papers that I like, that I thought might be helpful forsomeone who's thinking about who might have heard the term big data before, which usually

(12:34):
I think the promise that is being sold is that because you have such large amount of data,as I said, in the millions, scale could compensate
for a lot of the issues that we might have in traditional type of data we use,surveillance data that might be collected.
Of course, there are other properties for big data.
But I thought it might be a good place for us to start just introducing people to theconcept of big data.
If you talk about the, I think there's an article you published in the Annual Review ofPublic Health with colleagues thinking about equity and using big data and public health

(13:03):
research.
So If you can walk us through the five V's that people think about when they think aboutbig data and then the sixth
V that you introduced.
Ah, thank you so much.
That was a lovely team that I worked with.
It was just absolutely amazing.
It's led by Paul Wesson, who is an assistant professor at UCSF and a colleague.

(13:26):
and Margaret Handley, who is the senior author of it, a professor at UCSF.
And it was great to be able to kind of contribute that sixth V, because traditionally youkind of have these five kind of prongs of uh big data, right?
So you have velocity, which is kind of the speed of that data is collected.

(13:49):
um
and process and that's again similar to the online data like real near times and largeamounts of streams of data, which is then the volume, which is the size of the data.
So it's in terabytes ah or even larger now, right?
I think there's the value of it as well, which is how useful it can be in kind of decisionmaking.

(14:12):
There's the variety of it, which is the types of data that we have, right?
So it's like the structured unstructured multifactor.
I mean, it wasn't included at the time, but now it's multimodal, right?
You have images, you have audio that needs to be analyzed that you don't alwaysnecessarily get in traditional data.
um And then veracity, which is the authenticity and the kind of accountability of it.

(14:38):
um And so those are the kind of traditional five points.
And
I never like to say, again, I told you, I'm still a very big traditionalist in a lot ofsense.
And I don't want to ever say that big data or this type of data is going to replacetraditional data.
Like I said, I think qualitative interviews are so important.

(14:59):
But for instance, especially with this type of data, a lot of it is unstructured or a lotof it is really qualitative.
It's people speaking about their thoughts and feelings.
And when you have it in this such a large amount of size, you need these kind ofcomputational methods to be able to analyze that large amount of data.
So when we were kind of thinking about these kind of like five V's, we really didn't feellike there was this type of sixth V.

(15:25):
And so I came up with this idea of like that it should really be about, virtuosity, whichis this kind of
ethics of big data, right?
And so it's really honing in on these biases, but also the accountability of, you know,who is both producing kind of the data as in, maybe the platforms or even the people, but
also the positionality because everyone has a position, right?

(15:47):
Everyone has a stance.
And, another thing in this world is, it's become very political that you always come witha certain vantage point, right?
And you don't see someone else's because you either haven't had the experience.
But you need to understand that.
And so that was a really important point that we wanted to kind of add in this overallkind of review of the literature and to really, really try to emphasize this, this again,

(16:13):
idea of like, we cannot have, you know, this big data kind of analysis, without thisvirtuosity where we're always kind of thinking of, okay, what is, best
for everyone, including certain populations that are excluded, including ones that,especially with online data, there aren't, we don't always get to see or hear them.

(16:35):
But sometimes you can because a lot of populations also, again, as I said, that aretraditionally maybe underrepresented or have stigma or that don't feel like they're heard
or seen, but we can get that information online because people tell,
people talk a lot more online about their true feelings than they really do kind of inperson or to their physician.

(17:00):
So I think that this article kind of emphasizes that, uh but then also kind of goesthrough the kind of strengths and weaknesses and like, what we think kind of should be
done, which is again, is coming from more of that ethical lens that I think we should allbe having.
Yeah, it's interesting that you said that people talk more honestly online than they woulddo in person, which actually links to what I was wondering about as well.

(17:27):
This is great that we want to use big data and we need to think about equityconsiderations as we...
use big data in our our analysis.
But then do you think this approach of listening at scale could that have some tensionwith privacy issues?
And I'm still trying.
I think that's going to be I know that's oh this could be addressing a lot of the analysiswe do.

(17:47):
But I thought that was a thread that went through, I guess, as I was thinking about thisepisode of where do we draw the line of what how do we address the tension between?
Yes, we want to listen at scale.
We would love to get all the information we need.
But at the same time, we need to think about privacy concerns and
As you said, sometimes I wonder if people are more comfortable saying more honest thingsonline because they feel like they're not being judged in person.

(18:08):
So how do we navigate that?
will, I mean, this is a podcast and I'm always worried about saying my true feelings.
But I mean, again, these are my views.
But I get really frustrated sometimes because, you know, I mean, I do it too, where yourealize that all these companies, you're giving their data, your data to them.
It's just free for all.

(18:30):
And, you know, we click the yes box and they're analyzing our data all the time.
They're just not telling us they used to.
They used to kind of
publish on it and now they just kind of do it internally.
um Whereas, we go through like massive, IRB, you know, we, also have such restrictions onkind of the data that we're able to use, but also the data that we use is, we de-identify

(18:52):
it and we aggregate it.
So we're not seeing you literally like pointing out um people in that way.
I'm less worried about the like, researcher that's in the lab going through IRB, which isinstitutional review board,
And I do think, in other countries, for instance, like, Europe, like the European Union,they have a lot of restrictions in terms of what type of information companies are able to

(19:16):
capture and to be able to use.
And I think we could use kind of
more of those restrictions, for instance, in the United States.
um But in terms of, again, vantage point, but from my research area, I think it is, Imean, it's very beneficial in terms of the research sense to really understand, how people

(19:39):
are really feeling or the experiences they're really having, like, ah you know, one of thestudies that I did, you know, was, you know, looking at
Kind of the racism online after the term like Chinese virus was used for instance and kindof a big reason I did that was because I was so shocked at how many people really thought

(19:59):
that none of that racism was going on, right?
Until they actually see what people are saying, you know all the different like horrificterms that were being used for this population and then when people actually finally saw
it
then they say, okay, well, this is real, right?
I mean, unless someone is actually being able to describe it or you're actually able tosee it, it becomes more transparent and there, right?

(20:27):
But you don't actually believe it, unfortunately, if you don't see it, right?
Because we just say, well, these studies didn't say that they felt racism or like we did astudy on racism and guess what?
Everyone said, no, they're not racist.
Do you think that's, I mean,
is the social desirability kind of bias that we kind of have.

(20:48):
So I think those things are, I think that's actually so important.
um But you do have to have protections because there can be a lot of stigma.
I understand the fears around like healthcare um access and like insurance and havingeveryone kind of um know your potential disease or condition.

(21:10):
ah
But I will say at least for the research that I do, all of it is aggregated.
So we don't personally identify um people.
And I think it should remain that way unless people are willing to say who they are.

(21:32):
Which I think there's, it's shocking too, but a lot of people are kind of willing to putthemselves out there because they're willing to help others
for the overall benefit.
No, I agree.
And I also wonder about a generational shift where maybe people who...
who grew up today in a world where all of the information is out there, we might see, Iwouldn't say less concern of privacy, but people think about data differently.

(21:57):
Someone who was born maybe 60, 70 years ago might have a different view than someone whowas born 15 years ago and the type of world they're living in.
I actually do want to ask you about the practice that you use for your research, butbefore that, funny enough, before we start this recording, one of the AI tools I use, I'm
not going to name it, I opened it and it popped up saying,
Would you allow us to record your information for training data?

(22:20):
And this would mean instead of the 30 days we have now for storing your data, we wouldmove it to five years.
The interesting thing I thought they were not even using it for 30 days because I keepasking like over the past few weeks, every time I put in a document, I say, do you use my
data for training?
It says absolutely not.
So even the 30 day was a surprise to me, which I agree with you.

(22:40):
Then maybe companies have to have a very clear language because I think of my
off with someone who actually reads the fine.
print.
and I think I still realize that just I did not realize they still when they say we don'tkeep it, they're keeping it for 30 days.
So that's actually very interesting.
I don't know how we navigate a world where corporations and even government should takemore responsibility, making it very clear what type of data they have and what they're

(23:03):
using it for.
But then just moving to your practice, uh you've published a lot, a lot of high impactstudies and lot of journals that I think you use digital surveillance and using big data
from forecasting, I think, Lyme
disease to thinking about your anti-Asian sentiment early during the COVID-19 pandemic.
I think also with a group of researchers mapped global mobility using smartphone data.

(23:28):
maybe instead of just thinking about one paper that you've done, what is usually yourpractice?
do you think about the research question and then you think about the data you need or doyou think about the universe of the data that exists and then think about how would we use
that type of data to answer a question we might be interested in?
That's so, that's so such a challenge a lot of the time.

(23:49):
Um, and I do want to go back to, cause you, you opened up a can of worms in terms of, thedata kind of privacy issues, but so remind me, but so, I mean, I think, you know, I think
we're all very influenced by societal kind of trends, right?
Or not even trends, but events that happen, right?
So if it's COVID-19, you know, the mobility studies, everyone was working on it.

(24:13):
um I think a lot of the time too, you don't see from behind closed doors or how thesausage is made that it actually took five years to do that project.
By the time you try to acquire the data, get the funding, get the team, all that through.

(24:35):
So a lot of it is, uh it's kind of sometimes a juggling act.
You're doing multiple kind of projects at the same time.
I mean, if you look at my themes, I'd say, it's obviously around public health and the bigdata aspect, but the other one is like methodology.

(24:57):
So I really try to kind of think of the methods and try to evolve on that in the publichealth arena, which I think we are much slower to adapt to.
And so I'm always looking for, you know, what's kind of next in terms of the methods thatcan like add kind of more, more ability to take.

(25:24):
And like, I always say, this is like, ultimately what I do is, take the unstructured dataand make it structured.
Cause there's so much information out there that, what is it?
90 % of, for instance, communication is nonverbal, right?
And so we communicate, we communicate in that,
that nonverbal and structured world of like images, to memes to video, to again, podcastsand texts that is horribly messy and always in slang.

(25:52):
The joke is that, do you think as public health officials we can ask people to not useacronyms and not use slang so that we can better analyze the data and for it to be cleaner
because we have to spend so much time cleaning it?
But like that's ultimately what I want to do, right?
So I kind of really kind of start there on that aspect.

(26:12):
And then um I'm always, I think always at the back of my head is I'm always kind oflooking for ways to kind of uncover kind of hidden beliefs and behaviors that, you know,
we, traditionally haven't been able to hear, or find.
That was like ultimately and so that always kind of drives me, right?

(26:35):
I'm really interested in that type of dark underbelly of truthfully kind of like societyand that goes all the way from like how you're eating, how you're drinking, how you're
behaving, how you're, and what is causing those types of shifts.
ah And so kind of going from there, ah then kind of the...

(26:58):
different kind of projects emerge.
Yeah, so I have so many questions on this, especially, I don't know, maybe I'm actuallynow using this conversation as a way for me to think about using big data in my research.
But I have a lot more questions on this.
But before we move to those questions, I thought you wanted to mention also somethingabout the privacy issues.
the privacy thing.
It's so interesting because, you know, part of me, it's like you're the researcher, butyou're also the patient, you're also the public, right?

(27:25):
And you're all three of those hats, plus many more aspects about who you are, right?
And one of those aspects is also potentially being someone who is, you know, like aminority group.
Like I always think of it as, you people...
And I'm not an expert in this area.
I'm gonna say that first and foremost, but for instance, the genetic area where do youwanna get your genetic kind of DNA kind of analyzed and do you wanna know your family

(27:46):
history?
Do you wanna know everything about yourself for instance, right?
And the one aspect is the more information that you give, the larger the database goes,just in general, right?
And so, and in general, there's a huge problem across like different studies, right?
Including with women where there's not enough data, you know, because...

(28:07):
they don't donate it historically, right?
And so part of it of me as a researcher is thinking, okay, well, we need more of this datato be more representative, right?
So in my studies, it's, okay, ethics, right?
We need it to be generalizable and we need it to be uh ethical and representative ofpopulations.
But then as someone who is a minority, I don't want to do that because then it's...

(28:35):
then it's more identifiable, right?
Because there's so few of you.
And then I think, well, why do I want to give, for instance, my data to this geneticrepository and get information back on data that maybe is like five people?
I mean, it's not five people, but I'm saying for certain populations, it's five people.
So you're basing all of your information on five people.

(28:59):
And that's what we do a lot of the time.
Right?
And so the privacy issue is so multi-layered because it really depends on the populationthat you are from.
Right?
And data models are only as good as the data you give it.
And so if you're basing it on such a small number, it's such an issue, which is why I justam very hopeful of something like big data, where if there's so many people that we can

(29:26):
somehow be able to use that data to find better,
population kind of outcomes that are truly actually generalizable.
That's just what I wanted to say, because it reminded me when you were talking about, do Isave this data?
Well, no.
As a researcher, like, oh, I want to be so, I want the data to be contributed.

(29:50):
And yes, of course.
And then as an individual, no, go away.
I don't want any of my data.
struggling, have to think from that vantage point and that hat for sure because all of myfriends and family that aren't in the field say, oh, absolutely not.
But then again, then here they go.

(30:12):
My mom's on Instagram too, just giving her data wildly, freely, and then won't do any sortof any other kind of donation in the healthcare realm area.
But you do feel safer because of the
again because I think it's so many people.
uh But the other aspect, which I think is, I would say a little bit coercive is, are yougoing to say no to Google?

(30:39):
Are you going to say no to Instagram?
Are you going to say no to Twitter when those tools are, or LinkedIn, when those tools areso needed in a daily life?
Are you going to say no to Uber?
Who's saying no to Uber?
Hmm.
That's a good point.
Even if I say no to all of them, yeah, even if I decided to just boycott everything, Idon't know if I can ever say no to Uber.

(31:01):
That's a really good point.
Yeah.
Exactly.
It's a safety thing It's a saftey thing Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
is, uh, I think it's in New York times reporter that tries to not use her phone for likean entire, maybe few weeks to a month.

(31:23):
And one of the things is Google maps, you know, or, uh, you know, phoning someone, youknow, texting them to
tell them that they're late or to locate someone.
You have to be, which we all did, not talking to the next generation that doesn't have anyidea that you, when you said you were gonna meet someone, like you're waiting for 45

(31:43):
minutes hoping they were just late or if they're lost, I mean, what do you do?
You just go home and hope for the best.
yeah.
And you hope that they left you a voicemail.
I know.
I know.
And the other thing is you actually have to memorize a lot of phone numbers with this.
Yeah.
Exactly.
Yeah.
Exactly.

(32:04):
Yeah.
up to the age of what?
Maybe 12.
And I can remember every number before then, amazingly.
I agree.
So this is funny.
At BU, they have a survey.
And I felt really bad because they have this survey that's sent out to everyone.
But there were a few demographic questions they asked.
And I knew if I answered all of those, it's just I'm the only person who all this appliesto in the entire school.

(32:25):
So the researcher in me really wanted to just get the data in there because I would loveto have a
comprehensive understanding of what everyone in the school is doing.
But I also, as I said, the person will say, like they're asking about my ethnicity, whereI'm from, and a lot of other identifiers.
And exactly what you said, it is I knew at that moment, everyone who's going to read thecomments is going to know this is Salma, she wrote this.

(32:46):
So weirdly, or I guess not so weird, social desirability won.
And I think I was very, I toned down a lot of my answers exactly.
we want to ask people like, are you doing X, Y, and Z that's illegal and like, you know,disgusting or like, you know, bad for people.
You just can't, you know, say any of those things, but online it's quite shocking.

(33:10):
They really do.
And it's quite, and you know what?
It's so remarkable.
And I absolutely love it.
I'm not on any social media, I, I mean, I have no stake in it, but I just love Reddit.
I mean, Reddit is because it's,
anonymous so there's a problem with the issue of doing analysis for demographic issues butthere is no better platform than Reddit for finding out the truth.

(33:32):
It's just amazing, amazing.
agree.
I mean, maybe the winner here is just that if you Google anything, most of the time you'llsee that I say something Reddit.
I just want Google to give me what Reddit is saying.
That's what I've been first doing learning.
too, me too.
that's nice that we have that common.
Really me too.
I don't even look at the AI stuff.
I just just go straight to Reddit

(33:52):
Exactly.
Yeah.
This is funny what you said, because I also don't have social media.
So I also feel like a kind of a hypocrite because I did some sentiment of analysis usingTwitter data, but I stopped being on social media for a long time.
So maybe we'll talk at the end about what we say versus what we preach versus what weactually do in real life.
But really, going back to your research, and again, maybe I'll ask you to have aconversation after this, because now I want to learn more about what we can do with the

(34:19):
data that we have.
I think a few things came up that I wondered about.
So the first one is data validation.
So do you think about that?
think it is actually being sold oh from everything that you're saying about the importanceof sentiment.
And also, I think we mentioned social desirability a lot today.
But for those who are not really data nerds who use a

(34:39):
of survey data, social desirability is that usually when you ask someone something andyou're in their face or you call them or they think the person asking wants a certain
answer, people usually tend to give the answer they think others want to hear.
So I think actually sentiment analysis might answer some of those questions.
But maybe let's start with validation because then I think the other thing I was thinkingabout with the reverse of social desirability is whether

(35:07):
Do you think online sentiment could also be driven by seeing everyone jumping on a trend?
And does that affect what people also write about the trend?
Those are two questions, so maybe whatever you want to start with.
I'll pick the, go with the, I'll go with the validation first.
So again, is how we deal with it with this digital data is we usually, you know, like theLyme disease that you brought up, we usually compare it to the gold standard, which is a

(35:35):
traditional epidemiological source, right?
So with the Google searches for Lyme disease, we compared our model to the Centers forDisease Control
Lyme disease database, right?
ah And so that is our kind of comparison to see how close it is to be able to validate itfor its uh kind of representability, right, or um of the population.

(36:02):
But I think that's where I, you know,
I would like as an editor, which I don't know if I would be able to change it.
I would like to kind of change that aspect.
And I would like it to stand on its own a lot of the time, the digital data.
Cause again, I bring it up, but you know, we do that over, like a thousand surveys, and wetake people's information.

(36:25):
And this is a completely different data set and needs to stand on its own.
Right.
and I think people need to see that.
Uh, it's a, it's, it's a,
definitely a different representative population, know, skewed younger.
It actually has more minority populations, which I think is really good.
And so again, what's where I say is you need to merge the two of them, right?

(36:48):
And so, you know, for instance, like a study that I'm working on, and trust me, this is apassion project because it's, it bothers me, but it's so hard to get funding on women's
health, but in particular menopause, but just looking at, you know, the study that I'm
looking at right now is looking at kind of the EHR data and looking then at Reddit andlooking at menopausal kind of symptoms and treatments that people are doing, right?

(37:14):
And they're not lining up, right?
And it's because that's a gap.
That's a gap that the clinical records do not have just overall about menopause, butoverall about symptoms, right?
Because again, it's more of a challenge for people to talk about, you know, their sexualbehaviors and vaginal dryness and
a lot of it isn't asked in the clinical arena.

(37:39):
And so to, right now, I would like to look at it as now, is it a comparison to be able tosee the differences and say that, but I don't think it should just be a validation in that
way.
And I would like to move towards the fact that we're just looking at that data alone to beable to say,

(38:02):
hey, this is how people are feeling, and it's from a digital source versus this is howpeople online are, you know, that have online access or online are feeling about it.
It's like, no, no, no, this is what is being said online versus it being online people.
Cause we're, we're all using digital sources.

(38:22):
I mean, a majority of us are.
And again, I don't want to discount people who are not able to access it,
but a lot of people are, right?
And so again, as you mentioned in your intro so beautifully that this is our environment.
We think of it as like a physical space, but it is now the environment that we live andbreathe and think in and are people on.

(38:43):
And so we need to count that as real information.
And so that's what I'm always hoping the field is going towards, but it's...
It's taking a lot longer than I ever thought it would, but public health is a little bitslower that way and everyone else is doing it.
You have huge Fortune 500 companies, you have huge companies.

(39:04):
Pharma is really big in it.
They are online looking at patient documentation and symptoms and side effects online.
Large companies like Nike are analyzing social media data to understand their products andtheir reviews.
They're not doing surveys anymore.
Why are we not doing that for, you know, interventions or for products or for healthissues, right?

(39:24):
I mean, everyone else is taking that data and they're being very successful with it.
And yet, you know, I think that's where I say is like, sometimes I really do believe inthe traditional sense, but I think we just, like, it just needs to be, you know, you can
have, yeah, more like you can love traditional food.
Yeah.

(39:49):
That's a great metaphor.
Yeah.
Yeah.
I sometimes think we get really stuck and I'm there too in public health that, you know,in this type of the, the standard type of validation that we do and the standard type of
um representation that we see, which, you know, doesn't even work for politics anymore, tobe honest with you.

(40:09):
I agree.
I want, I I actually think I'm increasingly being sold just throughout this conversationwith thinking about big data as complementary and it's funny I actually have written a lot
about how we need to use big data.
it's, it's interesting, even as someone who worked on a commission on data science andusing big data for me, always think about the validation side and instead of thinking
about
This could just be complementary sentiments are important to measure other things comparedto just saying, no, we need someone in a clinic to validate what we were saying.

(40:35):
Quickly, and I think I think we're moving in multiple directions, but everything you sayjust makes me think about something that I'm working on slash thinking about because.
you were talking about trend next yes, do you want to move to that question?
oh
but before that I actually wrote a I was trying to write up a startup idea because I wasthinking no one wants to fund women's health.
Maybe if I write it in a very attractive way that makes convince a company that this issomething that's profitable.

(40:59):
Maybe they'll actually fund work on women's health because it's just fascinating to melike no one is willing to fund it.
But then I would love to talk about the trends.
I mean, truthfully, it makes me very upset And I think a lot of the time, you, you know,leaders make a really big difference in decision making.
and a lot of that.
In multiple areas, like I'm not just talking about healthcare and even government and techand everywhere.

(41:23):
If again, vantage point, if they're not experiencing menopausal symptoms, why would theykind of invest in it?
And then the other part is I think like we're there too where like profitability, I mean,it's really tough.
I don't want to discount myself either though, right?
It's, we take

(41:44):
I mean, not everyone, but sometimes we don't take care of our health as much as we doinvesting in like health products.
It was very different, right?
Like how much, how many times do you, and also the system too, like an ultrasound and Idon't think it's easy to get.
um But it just, em you know, it's like fertility for instance, for even so, know, thefertility of

(42:12):
the system, healthcare system in that grade, but privately it can be very good, right?
And so why do we have to pay for that when that, you know, people are constantlycomplaining about the fertility of, women going down, but then it's not free to do any of
it.
But then the other aspect of it again is like the products.
Like, I mean, we murder the, we murder the profit when it comes to, makeup and kind of

(42:37):
any sort of beauty products, but any sort of anti-aging regimens or like kind of thathealth like type of more like wellness, but not the kind of medical aspect of it.
ah And I think that's also historically because we couldn't turn to the medical system aswell as much.
And so we've had to rely on other kinds of sources.

(42:57):
em But I just haven't, I mean, people are trying, mean Halle Berry's trying, you know,Gwyneth Paltrow's trying.
I these are
big influencer people, like they people are trying.
There's a lot of investors, think it's Naomi Watts.
I mean, I'm naming a bunch of celebrities, but you know, I think they're trying and Ireally am grateful for it.
I mean, even Michelle Obama spoke about, it was so amazing, but she talked about goingthrough menopause, while being first lady, you know, and all she had to go through, how

(43:28):
she was getting these hot flashes and her makeup was dripping down and she didn't want to
do the speech while she was sweating because she would think that people were lying.
All these things that we have to go through that are never talked about.
And so then how do people know that we have an issue?
Right?
And so we've hidden it so long.

(43:49):
And I think that's what's great about I would say actually probably, you know, yes, it'sslightly your generation, but it's really the next generation that's really pushing those
conversations.
They have no fear in sharing.
And I think that is incredible.
Because I do believe that that is how we are gonna push forward is, and again, it'sonline, but to start, and don't get me wrong, it's very hard because there's been a lot of

(44:12):
stigma with it, but the more people that talk about it, ah the better it is.
And trust me, I do not wanna be the first person to talk about anything, but I do know.
in the trends.
Yeah.
When they talk about it, they will be captured in the big data trends and seeing thesentiment change.
Yeah.
So I'm hoping that this data analysis I'm doing on Reddit is going to talk about symptomsthat are never talked about in the clinic, and it doesn't mean they're not real.

(44:38):
yeah, yeah, yeah, yeah.
Trends.
Going back to trends.
I think, so with trends, people always say that.
What I say to people is that this online...
platforms, these online platforms, social media, they're both a way to captureinformation, but they're also a way that influences kind of information and behaviors.

(45:02):
I think that, you know, it's similar to being in, high school, and there's a bell bottomtrend that's going and maybe you don't like it, but you wear it anyways.
Does it then not, does it then not count as like this person is wearing bell bottoms,whether you believe it or not.
And then soon after,
maybe you really start just really liking bell bottoms, right?

(45:23):
And so I think that that's the influence kind of part is that if you contribute to thedialogue, even though you don't necessarily believe it, it's pushing that norm in that
direction, right?
Because other people are seeing it, right?
It's kind of like the same thing from traditional data, social desirability bias.

(45:45):
If you are saying you do not do this, then in the records,
You're not doing it.
You're not drinking, for instance, right?
Because you said, don't drink any, I don't drink, I don't smoke.
So in the records, it's not showing it.
So then everyone believes that's what everyone is kind of doing.
And it kind of maybe kind of sets that kind of belief.

(46:06):
And online, I think it's even kind of more powerful in that way where because of the otherone is we're on it so frequently and the whole health belief model, right?
Or no.
I'm getting it wrong.
Bandura's model.
Is it social belief model?
It's basically just this idea that

(46:27):
social learning theory.
Sorry, I also forgot about that.
And I taught a class about
okay.
So, but Ms.
Bandura, but basically, you know, you basically we mimic what we see, right?
That's how that's how children learn, you know, you see someone do something, that's howthey learn how to eat, that's how they how we learn out of language.
And so if we're exposed to it all the time, right, it's about, again, repetition, youstart to kind of believe it.

(46:52):
I see it with people all the time.
I see people's views have changed so much.
I mean, mine as well too.
Over time, when they used to think one thing and then all of a sudden, it's changed theway that they think about something and even behave.
I mean, diet's a really good one.

(47:13):
How many diet trends are there?
And then people follow them.
And people really follow them because it's like the new trend.
And so you might not believed it
before, but now you're kind of involved in it.
And then the same thing with, if you are making it okay to say hateful things at a certaingroup, that kind of normalizes something, right?
And again, we're like disassociating the like online world with the real everyday world,but we spend more time on the online world than we really do the offline world.

(47:40):
And I think we interact with a lot more people online on top of it too.
And that's where, especially peer networks and then
again, influencers that are that you maybe look up to, um that I think, again, really arekind of influencing and we've seen that, right?
What was it that, right, Kylie Jenner tweeted that she didn't like Snapchat and it lost, Ithink, $1.3 billion.

(48:05):
Elon Musk did the same.
He said the Tesla stock was too high and the stock went down.
So people are very influenced by people, right?
And so I think
Home depots now, or sorry, Whole Foods now um sells proteins as soon as I get it to thestore.
It's not even it's hidden somewhere else because now the trend in diet is that you have toeat as much protein as possible.

(48:27):
So now I go into Whole Foods and it's just you see it in your face.
And I think the New York Times actually wrote about uh just focusing on protein right now.
Yeah.
And because again, it started with the kind of an online trend, right?
The kind of carnivore diet, which by the way, I was way before everyone else because I dolove that.
But, no, but companies are monitoring that.

(48:50):
They're seeing that and it's influencing people's behaviors.
They wouldn't be changing out their stock if it wasn't influencing people's behaviors.
But again, we...
You know, again, it's the whole kind of validation where, oh, are people really sayingwhat they're doing?
Do you really think people are like, again, these surveys that you're getting from ahealth system that is taking care of your health, that you're already so vulnerable in,

(49:15):
are you really gonna tell them what you really do in your past time?
And I'm talking about small things, it's like, how many drinks did you have this week?
Like, do you smoke?
Do you do drugs?
Do you do illicit drugs?
Are you gonna answer that question,
legitimately, honestly, like I'm not saying you and me, I'm just saying in general, right?
And so again, like that's, I mean, I've done a lot of studies on that because I find thatextremely fascinating.

(49:38):
There's a whole kind of black market of things.
ah But I think that is an area where we could do more research on is kind of looking atthe list of market.
And again, it's aggregate, we're not picking out people, especially on somewhere likeReddit.
But we can really understand what the latest kind of drug trends are, what the mixturesare, what people are doing, how people are treating it.

(49:58):
And I just don't feel we do enough of it because it has to be validated with, it's thesurveillance part, This is kind of early warning system.
there's, back in the day, there's they used to, and again,

(50:20):
where they, you know, to be able to understand, if there was going to be an epidemic, theywould just look at, a few kind of sentinel hospitals, right, in certain areas, right, that
didn't, they didn't have enough resources in certain areas to be able to cover the entire,area or country.
So they would pick kind of a couple sentinel hospitals.

(50:41):
And if they started to have an outbreak, they knew that it was going to be massive, right?
And so like, can we do the same?
Can we look at a few influencers and studies have been done?
Can we look at a few people that are maybe these centers of the social network to see whatthey say and how that trend will change?
I look at kind of again, Joe Rogan and his influence, right?
And how that changes.
ah And again, that's where I would like to see that the field goes, kind of looking at howthings are shifting and then us trying to develop kind of interventions for that, you

(51:14):
know, online.
yes.
Okay, so I want to turn to AI now because I think this is it's the same thread, right?
AI just uses a lot more data in different ways.
Large language models mostly use prediction and machine learning to try to figure out whatto predict next, what we need or we're thinking about.
But what I really wanted to focus on here, because I do think it builds on your existingresearch is first of all, how is your research changing as you're thinking more about AI

(51:40):
and AI being more easily accessible to most of us?
I know a lot of people who work in machine learning
argued they've been doing a lot of this years ago before the LLMs were just available as achat box But at the same time I think now it's just getting a different understanding
among people like me So I don't know if having access to AI systems through chat boxactually change any of the work that you're doing or do you think it's just the exact same

(52:05):
thing that you're doing maybe on a larger scale.
And then I would like
to talk about your editorial role.
Yeah, think, uh so, I mean, I think that it's changed in the way that it's the LLMs,especially, these generative AI tools are so pervasive in both the public, but in the kind

(52:27):
of medical realm that it's hard not to be doing research kind of in somewhat in that area,especially when it relates to
for instance, ethics, right, is you have to be able to kind of test these, you test thesemodels to see if they, you know, have biases applied to different populations.

(52:49):
And so I think that I've done some research kind of in those areas, kind of to test those,to test those LLMs, like from, again, from the view from like an ethics kind of point of
view,
and seeing if they widen their narrow disparities and how they are with differentpopulations.
So think there's that aspect of it.

(53:10):
I think that the biggest issue with it is that field is moving so fast.
And so it's a challenge because the way that our research pipeline works is that the
peer review publications.
I'm not discounting that because replicability is very important and peer review is veryimportant.

(53:31):
But by the time you're kind of, you know, ready to publish the new kind of, new generativeAI tool has already come out.
Right.
So I think it's, it's having to have more speed in that area, both from the kind ofresearcher side, but also from the publication side.
uh So I think that's like a big kind of aspect about it.

(53:55):
um But I try not to change my focus on what one of my advisors, um Professor DavidWilliams, used to say is try not to go where the wind's blowing.
Keep your sails in one direction because you'll just get off course.
So have this northern star that's always there.

(54:15):
Again, my themes are still there.
And the area that I want to still focus on is kind of, again, this kind of like methodsarea and applying it to public health.
It's not necessarily developing like really kind of new methods.
Because I think, again, you know, the computer scientists in that area are not only arethey, you know, doing such advanced work, but their resources are just so massive.

(54:39):
We don't even have sometimes again, the storage and kind of data power to be able to dowhat they're doing, right?
And then we don't have the like level of teams that they have, right?
But I think applying those and having them answer public questions that we couldn't alwaysanswer is where I think like the sweet spot is.
So for instance, like I'm very interested in multimodal analysis, right?

(55:00):
So like all the data that again, that's unstructured data.
I really want to be able to now take that next step to be able to use it, to be able toagain understand population health,
understand social norms, understand those new modes of communication.
So again, always focusing on that blending with what I said, right, is this idea whereyou're taking that traditional epidemiology and you're merging it with computer science ah

(55:26):
is the sweet spot of where I like to be.
Plus, again, always having that type of ethical lens, which is the AIY checklist that weslightly briefly
talked about or if not at all.
Yeah, yeah.
No, no, I was actually going to ask about it because I read the article.
I thought it was really interesting.
I don't know if you might want to maybe explain to the listeners what you did in thatarticle and added to the show notes.

(55:50):
But I thought it was interesting because at least to me, as someone who hasn't written, I
present a lot and talk a lot about AI within public health schools because usually the tworeactions I get is people who are horrified or scared and don't want to use AI at all or
people who are just as dismissing this as something that is a hype that's gonna die soon.
And I usually just present and thinking like you can think about it, how do you use itreasonably without thinking that this is something that's gonna completely destroy the

(56:17):
Earth and maybe there's something in between.
So I thought that checklist was really helpful for a researcher
who wants to just maybe like dip their feet into the field of thinking about AI and publichealth or clinical medicine.
And then they can expand beyond that, but that's just like a great spot for them to start.
So maybe just tell us a bit about the checklist.
Yeah, so this kind of stemmed from ah the aftermath of the big data and kind of ethicspaper, the sixth V of virtuosity, because that one kind of took like more of a landscape

(56:51):
kind of perspective where this one people kind of wanted more like a solution type of basewhere, you know, well, what do I do?
How do I make these kind of assessments?
And so that was kind of the impetus for this paper.
And I know you talked about it with researchers, but
I would love people in tech and in industry to be able to use this.

(57:11):
So it goes through all these different case studies of using AI and represented bydifferent places around the world and talking about, again, a lot of the research was
focused on ensuring ethics.

(57:33):
ah of their work, but sometimes those gaps were missed, right?
And so, when you have kind of the limitation section, so then taking all of those and thentrying to kind of combine them together to kind of see, okay, well, if we were to develop
this kind of checklist, what would that look like?
And where would that be where we could actually be able to mark off yes and no?

(57:53):
Because again, coming from that perspective of ethics, you're thinking, okay, well, whatcould I do?
What could I not do?
And what did I miss?
And so then we kind of tried to put that all together in this type of checklist.
So I can run through it quickly, but it's kind of like looking at model adaptivity,accountability, the development teams, commercial interests, contextual adaptability,

(58:15):
accessibility, privacy, your favorite, transparency, ah know, targeted solutions and thengeneralizability, right?
And kind of going through and seeing like, have you, have you been able, are you able todo that?
Did you think about it at least?
I think is what was really important.
And it doesn't mean like you have to have all of them too, because sometimes it's just notfeasible.

(58:36):
But I think the point I'm trying to make is that like why I would like it to be part ofindustry so much is that, you know, having them really think about these aspects, I think
is really, it's just really important to ensure that like AI is really kind of fair andequitable.
That's interesting.
I like what you said about just thinking about it because we don't have to get into thisnow.
But some of them I read and I thought, excellent, I think that should just be on everylist.

(58:59):
And some of them I read and I thought, that could be a value judgment.
That's interesting because then it depends on what are the values that you're trying toadvance in this project versus not.
And maybe because I read a lot of population health science work uh and I think we think alot about equity efficiency trade-offs.
And it was interesting to me that for targeted, I think it was targeted

(59:20):
solutions you said you should prioritize equity which I think is great I would agree withthat but this is one of those what I thought oh this is interesting because I could see
one could argue if you're a commercial interest or you can a government depending on ourresources you might wanna
emphasize efficiency and then I know people always say then we'll get to equity later ornot But I thought that was that I thought it was a nice place to say those are things that

(59:43):
you need to think about a lot of them I think this should just be always there and some ofthem I thought you leave space for people to think about the different value Systems that
they're running with for that specific project
So then I have a quick question before we get into your role as an editor because I lovethis term I think it was on your website you said you worried about AI and cognitive

(01:00:03):
complacency and I thought that was great.
So do you want to tell us a bit about what do you mean by that?
Especially for people like me who use AI a lot?
I have to actually I should write an article about that so it doesn't get taken at thispoint but at least this is a record of it uh but uh I just really worried that you know,
basically what I'm trying to say is that we've become kind of lazy to kind of think forourselves Right, and I think in general, right?

(01:00:29):
I think you know, we are in a world where information is just kind of fed to us and we eatit, you know, without thinking where it came from.
Kind of like the food sometimes we eat and we got better at understanding, you know, wherethat came from, if it's good for us or not.
And before we were just kind of shoveling down food that we really didn't actuallyunderstand both again, the ethics of it, but also whether or not it was good for you or

(01:00:54):
not, or what the ingredients were.
We're much more conscious of that now.
But I'm worried at this stage with especially something like generative AI where theywrite
with such
you know, eloquency and such authority that you take that information immediately and youdon't think about that information that is being given to you.

(01:01:23):
And so when it comes to either making kind of decisions, and I know a lot of people aredoing it too, including, I haven't done it yet, but slightly of, you you enter in.
Ooh, do I choose this job or do I do this job?
Like, what should I ask this person or what questions should I ask this interviewer orwhat should I do that we're just letting a machine decide ultimately, right?

(01:01:45):
And so we're losing that kind of creativity and innovation.
And then we're becoming this kind of like collective hive mind at one point because, youknow, AI seems as though it's...
generating new information, but it really isn't.
It's kind of regurgitating information maybe in just different ways.
But then again, it's just regurgitation of information.

(01:02:06):
It's not the actual kind of synthesis and creation of, you know, new insights that youkind of just get sometimes just thinking alone by yourself.
I don't think we think alone by ourselves anymore.
We have to ask something and I do it too, you know, and then we don't, I do not do thisbecause
I again, we were trained so heavily not to, but is, you know, cited.

(01:02:32):
Where is this from?
You know, where is this information from?
Do you read the study?
Is it accurate?
Was the study done well?
But instead we kind of just take it the information as it is.
And so, but after a while, do you kind of get complacent and, you know, not thinking foryourself anymore.
And I worry about that, not just for the healthcare room, but for,

(01:02:54):
a lot of jobs, a lot of positions, like even when I read text, it's so similar now.
The words and everything and thoughts are so similar.
Yeah, I guess, and a lot of it is kind of, you know, when you get generative AI, getgenerative AI answers, they're kind of like really general, but then you realize they're
not actually that, like they're not really actually saying that much.

(01:03:16):
Do you remember when you used to write essays and you didn't really know the topic thatwell?
And then so you filled it with really fancy words and then you've gotten
I mean, it worked.
And so, I mean, I'm just worried about that if you don't actually, you know, if you don'tknow, if we don't train people, again, that's why traditional is really important.
If you don't train enough people with the content, you know, and really understand it,then how can they check if it's correct after a while?

(01:03:41):
I think some of the data actually support what you were saying.
There was a study, I don't know if it was an economist or from MIT, I'm not sure, butwe'll find it added to the notes where it showed that uh people are actually saying they
feel uh dumber using AI.
I don't know if that's,
that's actually the case or just the beginning of people adapting to AI.
I know Taylor Cohen, who is an economist, argued against this, but also is a biteccentric.

(01:04:03):
I also I think I saw recently someone writing an article from who used to teach at the UCsystem.
And he was saying he noticed that his students were just not really learning the material.
They're just getting A's in their assignments.
And then when they have the exam, they don't know what to do with the exam.
uh
as you said, just like just receiving the information that solves for the code, they werenot understanding how the code was created.

(01:04:24):
So they couldn't really translate it into the exam, which is not exactly what you'resaying.
But I think there are a few examples out there that are, should be a cause of concern forall of us.
But I also wonder if that should push us in thinking what should we do with theeducational system
Maybe we need to rethink how we teach people because as the traditional method works, butthen AI is just so pervasive everywhere that we really need to rethink what we do.

(01:04:46):
But anyway, so thinking about or what could I do if I want to publish on AI and JAMA +because you're the authority on it.
But maybe seriously first, I really wanted to hear a bit about the trends you're seeing ofpeople who are submitting papers to JAMA
under the AI banner, but also are there any papers you thought they were excellent thisyear that people should think about or read as they're thinking about how to incorporate

(01:05:12):
AI in their work?
Yeah, okay, so your first question was just overall, like, what does JAMAAI.org want?
ah So I will clarify that JAMA+ AI is a channel.
So it's a collective of all the journals, like in the JAMA network.

(01:05:34):
So JAMA, JAMA Ophthalmology, JAMA Neurology, JAMA Network Open, JAMA Pediatrics.
and it basically collects all the articles that are related to AI into a single channel.
So if you go there, you can see the latest AI publications that are across the network,right?
Instead of having to go to every single journal.

(01:05:57):
So it's just a one-stop shop to be able to get all that information.
Plus we have our JAMA AI
plus conversations and then we have other multimedia content, interviews, ah then newsarticles that are all on this kind of, again, single type of channel.

(01:06:18):
And so when you ask how do you kind of get published there, it's still submitting to oneof the JAMA network journals and then it'll get pushed to uh JAMA
AI.org.
In terms of kind of ah what we're seeing and what we kind of want to see, mean, we do see,you know, we saw a lot.

(01:06:43):
was definitely the year of large language models.
We saw a lot of, generative AI papers, some really cool ones that are in the realm of, youknow, the really influential article that had a of citations was, taking a look at how
generative AI responds to, uh patients and how they found that compared to, clinicians, uhthe generative AI tool was much more empathetic.

(01:07:13):
uh Which I don't I mean, I think empathy means you actually have to feel and I don't thinkAI is feeling right now.
So I don't think that it's true empathy that that's a point I will make.
But it got a lot of traction because could AI kind of be the replacement for clinicians?
And then that question of, well, would you want AI as clinician or not?

(01:07:37):
And that will bring me to one of the kind of papers that I did highlight as one of myfavorite papers.
So we got a lot of those.
And then we got a lot of kind of, again, testing of models.
you know, what we're trying to look for more is kind of prospective articles that arereally measuring patient outcomes, and making real kind of clinical changes in practice,

(01:08:02):
whether that be again, the health of the patient, or it can also be the health ofclinician as well, you know, is it going to improve efficiency?
Is it going to reduce, burnout, for instance, right?
I think
You know, the other thing too is, I mean, we are obviously very focused on, randomizedcontrol trials and trials in general.

(01:08:23):
That's always really important for us.
ah I also think the other area is, evaluation of AI tools that are existing or enhancingthem, not always having to necessarily develop things from scratch.
I think it's also really important, like how do you adapt it to maybe a certainpopulation, for instance, how we've seen some of those, which I think are very good.

(01:08:45):
We do have an article out that's with me and a few other authors led by um Alvin Lu, who'san editor.
um And um I would take a look at that kind of call to get an idea of kind of the articlesthat we are looking for.
um But yeah, I think, you know,

(01:09:07):
I think what I would like to see in general is I would like to see, I mean, I'm reallymulti-motor focused.
think image analysis is really cool.
And I think we have a lot to learn there.
I think it's very complicated.
And I'd like to see that kind of in the clinical realm.
There are some studies that have kind of looked at image data.

(01:09:28):
So one of them is by Ripart.
um
and it looked at a condition called focal cortical dysplasia.
And it's a brain abnormality that is very difficult to, it causes a hard to treat epilepsyand it can be difficult to spot on MRI scans, so patients can often go under diagnosed or

(01:09:52):
wait years for surgery.
And so,
what they did is they built an artificial intelligent tool called a graph neural network.
So it's basically like a computer model that looks at how different parts of the brain'ssurface connect.
And what was interesting is they found that over 60 % of the brain lesions thatradiologists had actually missed, ah the AI was able to find.

(01:10:16):
And so I think that that is very impactful clinically, right?
Because you can...
detect, you know, FCD earlier and more accurately, which means they can, you know, accesssurgery better and have fewer seizures and have better quality of life.
And then for, you know, doctors, the AI can provide this type of second set of eyes.

(01:10:39):
And so you can kind of see those subtle patient cues that can sometimes be missed, right?
And so like what I liked about this type of study is using again, methods part is reallyinteresting, right?
It's using this graph neural network, which is this type of AI that doesn't just analyzeimages pixel by pixel, but instead studies it like it's connected, like a connected

(01:11:01):
network, right?
Mapping the regions together.
And so it's particularly better at spotting these kind of like hidden abnormalities.
And so for me, like that,
again, having, again, that's really kind of the ver, sorry, the merging of what I wastalking to you about, right?
But it's more of like clinical and like computer science based, right?

(01:11:23):
Those type of graph neural networks are very much used in computer science and they'reusing it in the clinical realm on clinical data.
So, you know, that's where I really, that's where my interest is and what I would like tokind of be published.
Yeah.
anything that you see in an article that makes you think, this is definitely not for us,things that maybe pitfalls people should avoid?

(01:11:48):
Because I also think it is fashionable right now, I guess, for people to write about AI.
Yeah, think the biggest thing that I would say is that JAMA is still, it's a clinicaljournal, so our audience is very clinical.
ah And so ah it's not a computer science journal, In the way that, em well, the methodsare very heavy, and I do like methods.

(01:12:19):
The methods are very heavy.
It's very technical.
It's very long and our articles are not also formatted the same way as computer sciencejournals.
And so you have to really kind of understand that and look at how our journal isformatted, but also our audience.

(01:12:42):
So you have to speak to our audience that are our readers.
ah
kind of similar in a way to like a listener of podcasts, right?
You need to understand who's listening.
And so, you when you made the kind of definition of social desirability bias, that'sreally important because maybe not everyone knows that.
And so similarly, I just try to tell people that, you know, this very fancy computerscience method is written for computer scientists, but our audience is public health

(01:13:12):
researchers, clinicians, practicing clinicians,
Right?
And so I think that's a really important focus.
So that's like really what I see where, you know, it can be fantastic, but it has to bewritten for our audience because that's who we serve.

(01:13:33):
And so then speaking of podcasts, maybe quickly, you have a podcast
I think I listened to, I don't know how many hours of podcasts over the past few dayspreparing for this episode, but I wanted to ask you, what did you learn, especially
because you're interviewing physicians slash academics who are maybe have their topicexpertise.
But as you said, we, lot of us, almost all of us struggle with thinking aboutcommunication.

(01:13:55):
How do you get people to say something that would be interesting for a general audience oreven an audience who might be an academic or a physician, but not necessarily someone who
listens to podcasts
I mean, I would say to be as honest as possible, ah which is really challenging becauseeverything lives forever and it's so hard, but honesty really does shine through.

(01:14:22):
You can hold back, you can filter a little bit, but for it not to be rehearsed is theother one.
And it's really hard because you want to say the right thing and...
ah you want the questions beforehand, you want really sharp answers, but it comes acrossas rehearsed.

(01:14:47):
So I would say that's the advice that I try to give to myself if I'm doing an interview,if I'm an interviewee with you, but it's really hard because no one wants to be fired and
or canceled.
Yeah.
And so that's a real challenge.

(01:15:09):
And then, ah the other one for me is I think it's really important to be personal and totell a story that is personal to you.
Like have it relate to something that's in your life.
Makes it interesting for me to listen to because you're always going to have more emotionwith it.

(01:15:31):
Nothing sounds good when it's monotone.
And so if you can, you know, and I try to capture that when I'm an interviewer is to tryto capture a story that resonates with someone that brings out an emotion that's not just,
ah a factual regurgitation of the study that you did.

(01:15:55):
But that's hard.
That's really, it's actually really hard.
And you know,
Not everyone's made for camera and not everyone is made for voice either.
And that's okay if people want to just focus on the rest of the time.
It's okay too.
but some people are like, it's but you know, again, again, that's why you need that uniqueteams, right?
You need like communication experts, you need different people because, you know, peopleare brilliant.

(01:16:19):
People are brilliant in their their arena, right?
But it's, you know, it's really hard to pick up a baseball bat if you've never done itbefore.
And like, I'm one of them, right?
So, yeah, so.
Funny enough, I think that's why I tell people as well, is just we get better withpractice.
I think sometimes starting with a podcast, a small podcast is a good way to start and thenhopefully moving from there.

(01:16:42):
So maybe just, yeah, go ahead.
no, no, it's just it's also the other thing is it's so I think I'm hoping that I become abetter interviewee being an interviewer, right?
Again, different vantage points.
So you see what you want and practice is better.
ah But that that has also really helped, right?
JAMA AI conversations is helpful for me because I also learn from people.

(01:17:07):
I learn about the research area.
So it's fascinating to me to learn about people's studies, to learn about people'spersonal interests, why they did it, and the effect that they feel they're having is
something that I take with me all the time.
So that helps me in terms of, you know, ah when I'm interviewed kind of, you know, in theprocess and...

(01:17:33):
Yeah, you can ask the editor in chief of JAMA plus AI that I nerve racked all the time.
oh If I'm the interviewee and he's always telling me to calm down.
So, it's not like the nerves don't exist.
It's not like they don't exist.
They're just hidden behind this facade.

(01:17:55):
editing is an amazing tool as well.
Yeah.
Yeah.
Oh my goodness, bless the multimedia and editing team.
Cause that's the other thing when I tell, at least with us, it's not live, that we willmake you sound great either way.
And there's a chopping block.

(01:18:15):
And so it's okay too.
And you can repeat, you know,
You can repeat your answers if you really need to and you can chop stuff.
So that is a great relief, I will say, which doesn't always happen with a lot of podcasts.
No, I agree.
Funny enough, I actually I have one question before my last question, but uh I did aninterview two weeks ago with PBS and I actually think doing this podcast really helped.

(01:18:36):
I know the power of editing because I've done it now I know.
They're going to try to make me look as good as possible.
It's going to be fine.
Like, their job is nice this way, because sometimes it's not.
You know, you know, it's not a gossip column.
I'm not trying to make you look bad.
Um, but so, you know, hopefully if I look good, you look good too.

(01:18:56):
yeah.
Yeah.
Okay.
last question.
What do you think acting taught you that made you successful in public health research?
Oh gosh.
Oh my goodness.
And should we all go back and take acting classes or become actors?
Oh, hmm.

(01:19:17):
I'll have a two-pronged answer to that.
I think acting was just the best because man, oh man, do you face rejection like youwouldn't believe.
So that is just, although I will have to say nothing prepares you like academic rejectionbecause it's like, I mean, with...
at least back in the day more so and it's so much was it about a certain kind of look likethe only place where they can ask for a certain body type, certain height, certain race,

(01:19:43):
certain hair color and all that sort of stuff.
So you would be kind of like a lot of the time rejected on that aspect of it.
um And so I thought I had a thick skin, but like with academia, it sure, I mean, you, theygo back to the dawn of time and evaluate the entirety of you,

(01:20:04):
to like, you know, see if you're good enough for this role consistently.
You know, you just, can't believe how I still need reference letters.
It's absolutely shocking to me.
So you have to play nice in the sandbox all the time.
But the other one too is that uh you just, especially with academia, you always, I thinksimilar to kind of acting, always, you're only as good as your last role.

(01:20:29):
You're only good as your last publication.
So you always have to be kind of up in your field.
I think in that way.
But yeah, I mean, acting just taught probably the most truthfully, kind of the most thingswas about just overall rejection.
I've learned more now about it than I did kind of beforehand.
I would say though, I would say the best though is like being in the service industrytaught me a lot more.

(01:20:57):
Like I was a server, waiting on tables and
you know, you really kind of understand that perspective and you understand like customerservice and you understand to read people's emotions kind of well and what people want
because so much of is kind of, again, that's based on, you know, your tip ultimately.

(01:21:17):
And then you also just have a newfound kind of respect for that, like being on kind ofthat other side of things.
But no, nothing prepares you for academia.
think it does anything prepare you for academia.
I don't think anything prepares you for academia.
It is so different from every other industry.

(01:21:40):
It truly is.
It's so small.
It's again, the smallest minute thing.
You're critiqued for absolutely everything, both for your paper, both where you...
what degree you did it in.
Everything is just scrutinized so much.
And don't get me wrong, I do, there's so many aspects I really love about it, but I reallydon't think there's anything kind of like it.

(01:22:05):
It's very hard to, it's very unrelatable.
It is probably one of, again, I don't know the other areas and fields, but I'd say it'sone of the most challenging areas to relate to because there's so many different
complexity and layers, right?
Tenure and promotion and then publication and then like the balancing all of them, what itmeans like, are you really working?

(01:22:29):
But you're kind of at home just kind of like, you know, coding or doing analysis, butyou're also writing, but then you're also doing conferences.
Like it's a weird kind of mix of everything that you kind of get evaluated on.
You have to be everything all at once.
And then now we have podcasts, right?
But ah so that's like hard for...
people to understand kind of what you overall do, right?

(01:22:52):
I think the traditional academic of maybe like teaching is either the long since past orit's very seldom and far between, but now we do so much that I think it's, like I said,
there's nothing that prepares you for.
I It's an odd world.
And I did clinical medicine, I did policy, and then I came to academia.

(01:23:12):
And it's weird because you're getting paid to sit there and think, which is amazing.
But it also has all the things that you talked about.
So it's just weird.
It's a weird space.
Yeah.
uh
hard to measure that you're thinking.
It's like thinking about a new equation, right?
You spend how many years developing something and the classic is writing on thechalkboard, erasing it off.

(01:23:33):
But yes, you kind of take those notes, but really you come up with maybe one equation.
Its like, wait a second, you spent a decade making this equation or you spent 20 yearsmaking the like...
social belief or behavior model that has influenced everything.
What took you that long?
No, it really did.
It actually took a very long time to be able to synthesize, but it's hard to measurethinking.

(01:23:55):
Yeah, agree.
And then someone may become one comes one day and actually proves that what everythingthat you did was wrong, which is something.
Not
don't say that Salma.
oh
was speaking in the generic term of someone comes in, you have your theory for 20 yearsand then someone comes in and they're like, that's it.
But I agree.

(01:24:17):
I agree.
Yes.
Or the other one that is really bad is like poor Semmelweis, which people don't even knowis the person who, you know, developed hand washing, and he was thrown into a mental
institution and beaten to death, right?
Because he wanted people, doctors to wash their hands.
I mean, I don't want to be that person.
And then thought of as a genius after I've been dead,
you know, for 30 years, for all those other people who said the world was round, you know,when everyone thought it was flat and were hung or burned for, you know, and then they're

(01:24:46):
applauded.
I don't want to live that life either.
Like, I want, I would like a nice, like, middle ground where my ideas and theories areapplauded in my generation and time.
Please.
Please.
Yeah.
Yeah.
So always our last question here is the closing question is because we're calledComplicating the Narrative.

(01:25:07):
What is a narrative about AI, digital health or even public health that you really thinkneeds to be complicated?
I don't want to say I used to say that keeps you up at night, but then I was told maybe Ishouldn't ask academics what keeps them up at night because I don't want to hear the
answers.
But yeah.
I mean, I'll give the answer that you kind of brought up already, which is the cognitivecomplacency truthfully.

(01:25:29):
I think we need to think about that more.
Like, what is it actually doing to our society and our children?
Sam Altman said that there'll be no children that'll be smarter than AI because they'lljust be used to AI.
So does that mean they're not made for themselves?
Does that mean that AI is going to just do all the creating for us?
So I think that's like what
we need to really think about are we all just going to be in our own silos or all going tobe just one big kind of collective brain.

(01:25:59):
Well, that's our fun to end on.
Yeah, that's a good note, I hope.
Yeah, no, but that's a big one.
Yulin thank you so much.
This was so fun.
Thank you.

All Episodes

Episode Transcript

Popular Podcasts

Stuff You Should Know

Dateline NBC

The Male Room with Dr. Jesse Mills

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}What our digital lives reveal about health with Yulin Hswen

Episode Transcript

Popular Podcasts

.css-r6mb8g{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:1;overflow:hidden;}Stuff You Should Know

Dateline NBC

The Male Room with Dr. Jesse Mills

All Episodes

What our digital lives reveal about health with Yulin Hswen

Stuff You Should Know