July 9, 2025 • 13 mins
John Reites discusses the proliferation of self-proclaimed AI experts in the clinical research field and provides insights on how to identify genuine expertise. He emphasizes the importance of credentials, proven work experience, the ability to explain complex concepts simply, and a focus on responsible AI practices. Reites outlines four key criteria to evaluate AI experts, encouraging listeners to be cautious and discerning when seeking AI expertise.
Click here to message/text me your insights and ideas for future episodes
Thank you for joining Inclusion Criteria: a Clinical Research podcast hosted by me, John Reites. This is an inclusive, non-corporate podcast focused on the people and topics that matter to developing treatments for everyone. It’s my personal project intended to support you in your career, connect with industry experts and contribute to the ideas that advance clinical research.
Inclusion Criteria is the clinical research podcast exploring global clinical trials, drug development, and life‑science innovation. We cover everything clinical research to deepen your industry knowledge, further your career and help you stay current on the market responsible for the future of medicine.
Our episodes discuss current industry headlines, career tips, trending topics, lessons learned, and candid conversations with clinical research experts working to impact our industry everyday.
Watch on YouTube and listen on your favorite podcast app. Thank you for supporting and sharing the show.
Please connect with me (John Reites) at www.linkedin.com/in/johnreites or www.johnreites.com.
The views and opinions expressed by John Reites and guests are provided for informational purposes only. Nothing discussed constitutes medical, legal, regulatory, or financial advice.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_00 (00:00):
With AI everywhere, everyone seems to be calling
themselves an AI expert,especially in our industry.
And probably just like you, I'veseen multiple people who didn't
work in AI just a few months agonow promoting themselves as AI
experts or consultants.
So how do you know if someone isactually an AI expert or if
(00:21):
they're just aligning with thelatest trend to stand out?
And how do you...
verify if someone has theexpertise you want to support
your AI efforts in clinicalresearch.
Let's talk about it.
(00:46):
Now before we unpack today'stopic, I just wanted to spend a
minute to say thank you.
Thank you for spending your timewatching and listening to
Inclusion Criteria.
If we haven't met before, I'mJohn Reitz.
I'm a corporate entrepreneurturned digital entrepreneur with
my entire career focused onenabling clinical research for
everyone, everywhere.
(01:07):
Now, I fund and I created thisnon-corporate podcast to support
your career, connect with bothyou and industry experts, and
continue contributing to theactions and discussions that I
believe advance clinicalresearch.
Now I'm just a few months intonow my second podcast, and I'm
so grateful for your messages,for your comments, your ratings
(01:27):
of the show.
Because of you, InclusionCriteria has been moving up in
the charts in major podcastapps, on YouTube, and on
LinkedIn.
So thank you all for yoursupport.
I really appreciate it.
Now a few weeks ago, I took...
A very short call with an AIexpert who's now focused on
clinical trials.
Their pitch was that they hadsolved this complex challenge in
(01:49):
patient recruitment, and if theysolved it, it'd be a significant
step forward as an AI use case.
So I took the call, and I wasgenuinely interested in learning
about their solution.
Now, unfortunately, it doesn'tappear to solve the right
problem for me, but the red flagwas actually that the AI expert
didn't use several key AI termscorrectly.
(02:10):
They used every AI buzzword in abook, from LLM to a GenTech,
without really any clearexplanation of how the product
worked.
And when I asked what thedifferentiating or proprietary
features were, they would saythings like, that's proprietary,
and then shift and begin to sellme on another functionality of
the solution.
So these...
(02:30):
AI experts are popping upeverywhere and history does
often repeat itself because Isaw the same trend with
decentralized clinical trials asthe approach I worked more than
10 years of my career to helpdevelop, to advocate for, and to
regulate magically generated somany DCT experts in just a
couple months.
at the height of the pandemicwhen it was needed most.
(02:53):
So I think we all realize thatto be an expert in most things,
it takes years of focus, yearsof dedication and specific
actual delivery of work productin that area.
And I've been working with LLMsusing AI tools, vibe coding and
contributing technology featureswith AI in my company since
early 2022.
(03:13):
So I have some expertise in AI,but I don't consider myself an
AI expert yet.
And I'm sure like you, I'mcontinuing to put in the time
and actual work to get therewith a hyper focus on clinical
trial use cases that matter.
So in my opinion, it takes awealth of trial, error, and what
I would term successful mistakesto gain proven experience and to
(03:34):
call yourself an expert.
I think Werner Heisenberg, he'sa Nobel Prize winning physicist,
may have said it best when hesaid, an expert is someone who
knows some of the worst mistakesthat can be made in their
subject and how to avoid them.
So as we watch this AI boom inmotion, I'd strongly advocate
for you and anyone in ourindustry to take a measured but
(03:57):
rapid approach to adding someelement of AI expertise to your
skill set.
And to do that, you're going tohave to learn from others and
partner with those that haveexpertise in specific AI areas
and use cases.
So I believe there are fourcriteria you can use to verify
if someone is actually aclinical research-focused expert
(04:18):
in AI.
It's not an all-inclusive list,but I find these helpful in my
personal assessments, and I hopethey're helpful to you.
You can also apply thisverification process to yourself
if you want to be an AI expert,and apply it to any educational
training you may find on theinternets.
Just be careful of the YouTubeAI experts.
(04:39):
Now, the first criteria is aboutcredentials, and not all
credentials are important orcreated equal.
I'm sure there are now degreesforming in artificial
intelligence, but these are notactually what we're looking for.
Credentials come in a variety,so I would start by asking the
expert questions like, whatfield is your undergraduate or
advanced degree in?
(04:59):
And what degrees orcertifications do you hold
specifically in AI?
Do you have any peer-reviewed,not sales-related publications
around your work in AI?
Have you completed any industryvalidations with customers
around your AI solution?
Or do you have any case studiesor references from successfully
(05:21):
completed AI projects witheither a biopharma, a CRO, a
research site, or someone elsein our industry?
And if you're really bold andyou feel like you can ask this
on the call, you can also ask anexpert if they've been cited or
partnered with other AI expertsand leaders in the field.
Now, I'm typically looking for adegree or some kind of an
(05:43):
advanced degree in computerscience and machine learning,
maybe in data science orsimilar, if they're involved in
the actual coding or developmentof the solution.
So I would start here, but no,this is not how I would write in
or write off an AI expert.
If they've worked at a larger AIcompany, such as OpenAI or
(06:05):
Anthropic, or if they've workedon an AI project within a
company that really hadcredibility and used the tools
publicly, I do think thismatters more than anything else
you may be able to find abouttheir credentials on the
internet.
Now, on the flip side, if an AIexpert openly refers to
themselves or uses terminologyon their social platforms like
(06:28):
AI visionary, AI luminary, AIfuturist, mentor to the stars,
or similar, whatever those termsare, they may be right.
And this may be an accuratereflection of their expertise.
But openly, in my experience,it's typically the opposite.
So this is something to becautious with and look out for
(06:49):
when you're trying to select anexpert to partner with.
Now, if after all thesequestions, they have little to
none of these credentials, Iwould consider passing.
on working with or learning fromthis AI expert.
The second criteria, and what Iwould consider arguably the most
important, is actual proven workexperience.
(07:10):
The ability to speak in depth ona specific use case that solved
a specific challenge for aspecific type of client is what
makes me feel confident in anexpert.
Experience also has aninteresting way of helping
experts to educate people oncomplex activities with the
(07:31):
utmost simplicity.
Not too long ago, I had anopportunity to tour a facility
with a working nuclear reactor.
And as cool as that was to seethe reactor, what actually
really impressed me was theability for the experts in the
field to explain the complexityof nuclear power in a way that I
couldn't even understand.
(07:51):
They didn't just manage thereactor.
They knew every component of itso well that they could explain
how each of the parts theymanaged worked at mostly a high
school grade level.
Unpacking someone's AI expertiseand seeing an expert light up
and explaining how they're usingAI and going truly in depth on
(08:13):
how the solution works and whatyou figured out is such a good
way to learn about expertise.
So if this is what you want tolearn about, I would start with
asking questions like, could youexplain to a high schooler how
an LLM or a large language modelworks?
How many specific projects haveyou supported with this AI
(08:34):
solution?
Could you show me the liveplatform and how the AI works
for your clients?
Could you provide a case studywith specific data showing where
the AI solution performed well?
And on the flip side, could youplease provide a case study with
specific data showing where thesame solution underperformed?
(08:57):
Other questions include, how didyou support the data
requirements to enable your AIsolution?
Or, How do you review,reference, and clean the data
that is powering yourtechnology?
And for extra points, do youhave a GitHub repo with any of
your AI projects in it?
If you get vague answers, blankstares, or if you feel confused
(09:18):
by the AI expert's responses,this should be a key red flag
for you.
One of my favorite words to usein business is measured.
And it's the third criteria.
The ability to be confident,direct, and maybe even a little
cocky can always be balancedwhen an expert is measured.
This ability to know a problemand its solution so deeply while
(09:41):
also being clear about both thepositives and negatives is so
important.
Real experts see positives andnegatives as just data points
and not a reflection on theirpersonal ability or their
personal performance.
So if someone only talks abouthow amazing AI is without
mentioning any problems orlimitations, it's a huge red
(10:04):
flag.
And if you hear anything similarto AI will solve all your
problems or we'll just automatethis system so it works
automatically for you, or ifthey use the word automagically,
I would consider ending the callearly.
AI experts understand that theverification, the math, and the
(10:25):
process behind their solution.
They don't need to reciteformulas for you, but they
should grasp beyond thefundamental AI concepts and be
able to explain them in detail.
A real AI expert will tell youabout the successes and the
challenges of their AI solution.
And if they don't, this is alsoa potential red flag.
(10:45):
And I'd recommend asking themquestions like, What gaps does
AI have today and how does itfail?
What parts of AI are you not anexpert in?
What's the specific return oninvestment I should get with an
AI solution before scaling it?
Have you ever removed AI from asolution because it did not
improve it?
(11:06):
Or what's the biggest limitationof current AI technology in your
field?
You know, real experts areexcited to talk about problems
because that's where theinteresting work happens.
Fourth and final criteria isabout ethics.
If an AI expert doesn't mentionthe term ethics, bias, or
safety, I'd flag that as aconcern.
(11:28):
Now, this is a massive topic andnot something I'm gonna do
justice to in this shortreference, but at a high level,
Real AI experts are obsessedwith responsible AI practices
because they've seen how thingscan personally go wrong.
They also understand that AI isa specialized field with many
focus areas.
Nobody is an expert ineverything AI, and we should all
(11:52):
be concerned with safeimplementation of this tool in
society.
So if the expert doesn't bringthis topic up, I would advise
asking questions like, how areyou managing the safety and
privacy of your AI solutions?
How do you manage AIhallucinations?
What about AI and its growth?
is the most concerning to youtoday?
(12:13):
Or what are the constraints weneed to manage when we're
implementing our own AIsolution?
How should we think about biasin our AI solutions?
These four criteria should be ahelpful start in your journey to
identify and learn from AIexperts in clinical research.
So please remember, real expertsexplain clearly.
They discuss limitations.
(12:34):
They've built actual systems andthey can get specific about
their work.
Don't be impressed by buzzwords,by titles, or by grand promises
about what AI can deliver.
Be impressed by clearexplanations, honest discussions
of challenges, and concreteexamples of real work an expert
(12:56):
can show you, not just tell you.
So have you encountered any AIexperts that generated similar
red flags?
Do you have a list of anycritical questions to add to our
list?
Please comment.
or send them to me on LinkedInif you do.
Thanks for listening and lookforward to next time.
With AI everywhere, everyone seems to be calling
themselves an AI expert,especially in our industry.
And probably just like you, I'veseen multiple people who didn't
work in AI just a few months agonow promoting themselves as AI
experts or consultants.
So how do you know if someone isactually an AI expert or if
(00:21):
they're just aligning with thelatest trend to stand out?
And how do you...
verify if someone has theexpertise you want to support
your AI efforts in clinicalresearch.
Let's talk about it.
(00:46):
Now before we unpack today'stopic, I just wanted to spend a
minute to say thank you.
Thank you for spending your timewatching and listening to
Inclusion Criteria.
If we haven't met before, I'mJohn Reitz.
I'm a corporate entrepreneurturned digital entrepreneur with
my entire career focused onenabling clinical research for
everyone, everywhere.
(01:07):
Now, I fund and I created thisnon-corporate podcast to support
your career, connect with bothyou and industry experts, and
continue contributing to theactions and discussions that I
believe advance clinicalresearch.
Now I'm just a few months intonow my second podcast, and I'm
so grateful for your messages,for your comments, your ratings
(01:27):
of the show.
Because of you, InclusionCriteria has been moving up in
the charts in major podcastapps, on YouTube, and on
LinkedIn.
So thank you all for yoursupport.
I really appreciate it.
Now a few weeks ago, I took...
A very short call with an AIexpert who's now focused on
clinical trials.
Their pitch was that they hadsolved this complex challenge in
(01:49):
patient recruitment, and if theysolved it, it'd be a significant
step forward as an AI use case.
So I took the call, and I wasgenuinely interested in learning
about their solution.
Now, unfortunately, it doesn'tappear to solve the right
problem for me, but the red flagwas actually that the AI expert
didn't use several key AI termscorrectly.
(02:10):
They used every AI buzzword in abook, from LLM to a GenTech,
without really any clearexplanation of how the product
worked.
And when I asked what thedifferentiating or proprietary
features were, they would saythings like, that's proprietary,
and then shift and begin to sellme on another functionality of
the solution.
So these...
(02:30):
AI experts are popping upeverywhere and history does
often repeat itself because Isaw the same trend with
decentralized clinical trials asthe approach I worked more than
10 years of my career to helpdevelop, to advocate for, and to
regulate magically generated somany DCT experts in just a
couple months.
at the height of the pandemicwhen it was needed most.
(02:53):
So I think we all realize thatto be an expert in most things,
it takes years of focus, yearsof dedication and specific
actual delivery of work productin that area.
And I've been working with LLMsusing AI tools, vibe coding and
contributing technology featureswith AI in my company since
early 2022.
(03:13):
So I have some expertise in AI,but I don't consider myself an
AI expert yet.
And I'm sure like you, I'mcontinuing to put in the time
and actual work to get therewith a hyper focus on clinical
trial use cases that matter.
So in my opinion, it takes awealth of trial, error, and what
I would term successful mistakesto gain proven experience and to
(03:34):
call yourself an expert.
I think Werner Heisenberg, he'sa Nobel Prize winning physicist,
may have said it best when hesaid, an expert is someone who
knows some of the worst mistakesthat can be made in their
subject and how to avoid them.
So as we watch this AI boom inmotion, I'd strongly advocate
for you and anyone in ourindustry to take a measured but
(03:57):
rapid approach to adding someelement of AI expertise to your
skill set.
And to do that, you're going tohave to learn from others and
partner with those that haveexpertise in specific AI areas
and use cases.
So I believe there are fourcriteria you can use to verify
if someone is actually aclinical research-focused expert
(04:18):
in AI.
It's not an all-inclusive list,but I find these helpful in my
personal assessments, and I hopethey're helpful to you.
You can also apply thisverification process to yourself
if you want to be an AI expert,and apply it to any educational
training you may find on theinternets.
Just be careful of the YouTubeAI experts.
(04:39):
Now, the first criteria is aboutcredentials, and not all
credentials are important orcreated equal.
I'm sure there are now degreesforming in artificial
intelligence, but these are notactually what we're looking for.
Credentials come in a variety,so I would start by asking the
expert questions like, whatfield is your undergraduate or
advanced degree in?
(04:59):
And what degrees orcertifications do you hold
specifically in AI?
Do you have any peer-reviewed,not sales-related publications
around your work in AI?
Have you completed any industryvalidations with customers
around your AI solution?
Or do you have any case studiesor references from successfully
(05:21):
completed AI projects witheither a biopharma, a CRO, a
research site, or someone elsein our industry?
And if you're really bold andyou feel like you can ask this
on the call, you can also ask anexpert if they've been cited or
partnered with other AI expertsand leaders in the field.
Now, I'm typically looking for adegree or some kind of an
(05:43):
advanced degree in computerscience and machine learning,
maybe in data science orsimilar, if they're involved in
the actual coding or developmentof the solution.
So I would start here, but no,this is not how I would write in
or write off an AI expert.
If they've worked at a larger AIcompany, such as OpenAI or
(06:05):
Anthropic, or if they've workedon an AI project within a
company that really hadcredibility and used the tools
publicly, I do think thismatters more than anything else
you may be able to find abouttheir credentials on the
internet.
Now, on the flip side, if an AIexpert openly refers to
themselves or uses terminologyon their social platforms like
(06:28):
AI visionary, AI luminary, AIfuturist, mentor to the stars,
or similar, whatever those termsare, they may be right.
And this may be an accuratereflection of their expertise.
But openly, in my experience,it's typically the opposite.
So this is something to becautious with and look out for
(06:49):
when you're trying to select anexpert to partner with.
Now, if after all thesequestions, they have little to
none of these credentials, Iwould consider passing.
on working with or learning fromthis AI expert.
The second criteria, and what Iwould consider arguably the most
important, is actual proven workexperience.
(07:10):
The ability to speak in depth ona specific use case that solved
a specific challenge for aspecific type of client is what
makes me feel confident in anexpert.
Experience also has aninteresting way of helping
experts to educate people oncomplex activities with the
(07:31):
utmost simplicity.
Not too long ago, I had anopportunity to tour a facility
with a working nuclear reactor.
And as cool as that was to seethe reactor, what actually
really impressed me was theability for the experts in the
field to explain the complexityof nuclear power in a way that I
couldn't even understand.
(07:51):
They didn't just manage thereactor.
They knew every component of itso well that they could explain
how each of the parts theymanaged worked at mostly a high
school grade level.
Unpacking someone's AI expertiseand seeing an expert light up
and explaining how they're usingAI and going truly in depth on
(08:13):
how the solution works and whatyou figured out is such a good
way to learn about expertise.
So if this is what you want tolearn about, I would start with
asking questions like, could youexplain to a high schooler how
an LLM or a large language modelworks?
How many specific projects haveyou supported with this AI
(08:34):
solution?
Could you show me the liveplatform and how the AI works
for your clients?
Could you provide a case studywith specific data showing where
the AI solution performed well?
And on the flip side, could youplease provide a case study with
specific data showing where thesame solution underperformed?
(08:57):
Other questions include, how didyou support the data
requirements to enable your AIsolution?
Or, How do you review,reference, and clean the data
that is powering yourtechnology?
And for extra points, do youhave a GitHub repo with any of
your AI projects in it?
If you get vague answers, blankstares, or if you feel confused
(09:18):
by the AI expert's responses,this should be a key red flag
for you.
One of my favorite words to usein business is measured.
And it's the third criteria.
The ability to be confident,direct, and maybe even a little
cocky can always be balancedwhen an expert is measured.
This ability to know a problemand its solution so deeply while
(09:41):
also being clear about both thepositives and negatives is so
important.
Real experts see positives andnegatives as just data points
and not a reflection on theirpersonal ability or their
personal performance.
So if someone only talks abouthow amazing AI is without
mentioning any problems orlimitations, it's a huge red
(10:04):
flag.
And if you hear anything similarto AI will solve all your
problems or we'll just automatethis system so it works
automatically for you, or ifthey use the word automagically,
I would consider ending the callearly.
AI experts understand that theverification, the math, and the
(10:25):
process behind their solution.
They don't need to reciteformulas for you, but they
should grasp beyond thefundamental AI concepts and be
able to explain them in detail.
A real AI expert will tell youabout the successes and the
challenges of their AI solution.
And if they don't, this is alsoa potential red flag.
(10:45):
And I'd recommend asking themquestions like, What gaps does
AI have today and how does itfail?
What parts of AI are you not anexpert in?
What's the specific return oninvestment I should get with an
AI solution before scaling it?
Have you ever removed AI from asolution because it did not
improve it?
(11:06):
Or what's the biggest limitationof current AI technology in your
field?
You know, real experts areexcited to talk about problems
because that's where theinteresting work happens.
Fourth and final criteria isabout ethics.
If an AI expert doesn't mentionthe term ethics, bias, or
safety, I'd flag that as aconcern.
(11:28):
Now, this is a massive topic andnot something I'm gonna do
justice to in this shortreference, but at a high level,
Real AI experts are obsessedwith responsible AI practices
because they've seen how thingscan personally go wrong.
They also understand that AI isa specialized field with many
focus areas.
Nobody is an expert ineverything AI, and we should all
(11:52):
be concerned with safeimplementation of this tool in
society.
So if the expert doesn't bringthis topic up, I would advise
asking questions like, how areyou managing the safety and
privacy of your AI solutions?
How do you manage AIhallucinations?
What about AI and its growth?
is the most concerning to youtoday?
(12:13):
Or what are the constraints weneed to manage when we're
implementing our own AIsolution?
How should we think about biasin our AI solutions?
These four criteria should be ahelpful start in your journey to
identify and learn from AIexperts in clinical research.
So please remember, real expertsexplain clearly.
They discuss limitations.
(12:34):
They've built actual systems andthey can get specific about
their work.
Don't be impressed by buzzwords,by titles, or by grand promises
about what AI can deliver.
Be impressed by clearexplanations, honest discussions
of challenges, and concreteexamples of real work an expert
(12:56):
can show you, not just tell you.
So have you encountered any AIexperts that generated similar
red flags?
Do you have a list of anycritical questions to add to our
list?
Please comment.
or send them to me on LinkedInif you do.
Thanks for listening and lookforward to next time.
Popular Podcasts
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.