August 12, 2025 • 31 mins
In this episode, Tom Caravela is joined by Lusine Kodagolian to discuss how to measure the value of medical affairs. They explore the SMART framework, emphasizing its application in measuring scientific ROI and share of scientific voice. The discussion highlights the balance of various metrics and how to build a business case for SMART metrics, especially for smaller or early-stage companies. The conversation covers insights versus impact measurement and the implementation of new strategies. Consistency, trends, and feedback in metric analysis are underscored as essential elements. The episode concludes with actionable steps for integrating these strategies in medical affairs.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:01):
Hey, guys.
Welcome to the podcast.
I am very excited to have another, valuediscussion.
I know that this is such an important topic forMSLs and for medical affairs.
And I have Lucine with me.
Lucine, welcome to the podcast.
Are you excited?
Very.
(00:22):
Thank you so much for having me.
I'm gonna let you let's guys.
So the title of this episode is frominteractions to impact, how medical affairs can
prove its value.
But, Lucine, why don't you do an introductionfirst so that you could tell everybody who you
are, and then we can we can just kinda jumpright into it.
(00:44):
Okay.
Thank you again, Tom.
I am really excited to be here.
I'm the founder of Cypherio, an AI enabledplatform that helps medical affairs teams
measure their true impact.
My entire background is actually in measurementand communications and analytics.
I've spent decades helping corporatecommunications teams at some very large
(01:05):
organizations measure the impact of their work.
And by very large, I that includes thePentagon.
I've actually spent a few years helpingPentagon measure their impact of
communications, and it's actually aninteresting story because it led to, as you can
(01:26):
imagine, several very interesting projects andseveral very interesting rumors.
When we moved to California, my daughter'sfriends were convinced that I'm a Russian spy.
I didn't exactly deny it, but I found out aboutit from one of the parents at a drop off.
So
maybe quite interesting.
(01:46):
Funny.
You I could see that.
I mean, you look the part.
You sound the part.
You get the whole thing down.
I'm going with that.
I'm gonna say I had a Russian spy on thepodcast.
Well, I'm I'm Armenian, so I don't even knowwhere that came from.
But somehow, somewhere, they they must havemade that connection.
Yep.
But in all seriousness, I've spent much of mycareer helping teams move beyond vanity metrics
(02:09):
and to measure their real impact, and that'swhat we're bringing to medical affairs and
hoping to discuss here at this podcast.
Well, the the one thing that I I think thereason I'm excited about this conversation, the
way it's gonna be a little bit different is weare gonna talk about impact.
We're gonna talk about metrics numb notmetrics, but analytics and numbers and not, you
(02:33):
know, measurement, not just concept.
So but before we get into that, I do have aquick announcement.
I wanna remind everybody to register for FiercePharma Week.
It's coming up very quickly, September, andthat's in Philadelphia at the Convention
Center.
This used to be Mass East.
(02:54):
It's called Fierce Pharma week now.
It's a much larger event because while it stillhas medical affairs, it also has PR and
communications, business development andlicensing, and pharma marketing.
So there's there's different functional areas,which is kinda cool because it's a bigger event
so you get to meet a lot of different people,but medical affairs is still its own its own
(03:19):
thing.
So for more information, go to Fierce PharmaWeek.
Just Google it.
When you register, use the code MSLTALK, nospaces, to get 25% off.
And I look forward to seeing you out therebecause I'm gonna definitely be there.
So, definitely register.
So, Lucine, let's get into this.
(03:41):
Why do you believe medical affairs, hashistorically struggled to quantify its value,
obviously, compared to commercial teams?
Right.
So, this is a problem that hits, severaldifferent depart departments, and I just
mentioned corporate communications a minuteago.
(04:02):
But the essential question is, how do youmeasure the value of roles that don't bring in
revenue?
As we know, from just our experience of workingwith medical affairs teams, medical affairs was
not originally built for ROI tracking.
It kind of evolved from that support functionfocused on scientific integrity.
(04:25):
The impact of medical affairs, the way we seeit, is often long term and indirect, which
makes it very difficult to measure usingtraditional business metrics.
Commercial teams have long had theinfrastructure and this data driven mindset.
Medical affairs have also operated with adifferent purpose, the commercial than
(04:48):
commercial.
Right?
We, with medical affairs, we focus on advancingthe scientific narrative, the scientific
integrity, whereas commercial teams are builtall around revenue and quotas and measurable
outcomes.
So to give an example, medical affairs canspend many, many hours.
(05:12):
The teams can spend many hours meeting withKOLs, and, eventually, the result may, be the
inclusion in guidelines, but that may take acouple of years.
That's not exactly something that shows up on apretty dashboard.
Whereas commercial teams can very clearly say,we're in this campaign, and we saw this many
(05:33):
and we saw an increase in prescriptions or, youknow, this percentage change.
So it's it's a little it's much more clear cutand easier to measure than what we see for
medical affairs.
The what's tricky about the work of medicalaffairs also, it's it's very subtle.
You're helping shape the scientificconversation, the scientific narrative,
(05:57):
supporting evidence generation.
But, again, as I mentioned earlier, that's notsomething that you can easily capture in a
dashboard.
Yeah.
Well and it's just I mean, it's it's so easyfor commercial.
There's like you said, there's just so manyways to track it.
So how do you it's so hard to track it in themedical affairs world.
(06:19):
It's it's not as as easily cut and dry becauseyou can't just look at sales and say, okay.
Well, we did this, and then that influenced thenumber of sales that we had.
And there's the return on our investment andour time.
So let's talk about when we talk about, some ofthe ways that MSLs are being tracked, like
(06:42):
metrics, for example, would you say there's anyunintended consequences of relying too heavily
on what we see is a lot of activity basedmetrics, like number of interactions.
Can we talk about that?
Yeah.
Absolutely.
And on the surface, that looks great.
Right?
You capture how many meetings you had, howfrequently you meet with your KOLs in your
(07:07):
territory or HCPs in your territory.
So, it's a great metric, again, on paper and ondashboards.
But in essence, when you are only measuringthese activity metrics, you are incentivizing
volume.
So instead of doing better, you are focusing onjust doing more.
(07:28):
And this can lead to this false sense ofefficiency that, oh, we've had this many
meetings, so we're doing great.
But what if those meetings did not have theimpact, the intended impact that you're trying
to achieve?
For example, let's say an MSL meet met with 50HCPs in a month or a 100 HCPs, and that's what
(07:51):
we see more often than not.
But, again, what does that mean?
Did these meetings lead to any meaningfulchange?
Did the HCPs walk away with something that youcan actually follow-up with and see whether it
had the impact you intended?
Where did they understand the key messages, thekey data points that you were sharing?
(08:14):
You don't know.
Did you meet with the right people?
Essentially, you're busy, but you're notnecessarily sure if you're effective or not.
The activity metrics also have a tendency tocreate this illusion of productivity.
You show high numbers to leadership.
They feel like the goals are accomplished.
(08:34):
But in reality, as I said, it just creates itsfull sense of effectiveness and accomplishment
when in reality, you may not have accomplishedthat.
Now with that being said, there are certainterritories that are it depends.
Right?
If you're in a smaller territory and you'reearlier in the product life cycle, it makes
more sense to focus on the number of meetingsbecause you may not have access to top KOLs or
(09:00):
top institutions.
And so, therefore, the the frequency ofmeeting, the frequency of these touch points,
the length of those meetings matters becauseyou're trying to introduce a new concept.
You're building awareness around the disease.
And so the more meetings you have, the morebeneficial it is for your team.
(09:20):
So at the end of the day, I would say thebalance is determined by your strategy.
If you tie your metrics to your strategy, thenthe decision of whether you're focusing on
quantitative or qualitative or what the rightcombination of the two is, it becomes a lot
more obvious.
Well, let's I wanna talk about like, let'slet's get into that piece of it because I know
(09:44):
when you and I spoke Mhmm.
Originally, you had mentioned the SMARTframework.
So you have this SMART framework.
Mhmm.
Can you break that down for me?
Like, so how can medical affairs apply thecomponents of what your SMART framework is, and
how can that help them meet their goals?
(10:05):
Yeah.
Absolutely.
So the SMART framework is what the corporatecommunications teams adopted a while ago, I
would say about fifteen years ago.
And I don't mean all of the teams, but the onesthat wanted to show real impact.
Because, initially, all these teams weresimilar to medical affairs.
(10:25):
We're measuring media impressions, numb thenumber of whips website clicks, number of
visitors, how many press releases they issued.
And while that looked great on paper, again,were all vanity metrics that didn't really show
if they were accomplishing the intended purposeand if they were having the right impact.
(10:45):
So they adopted what they call the smartframework, measurement framework.
And the biggest advantage of this smartframework is that it forces you to be very
clear and to break down the goalsystematically.
Essentially, if we walk through it, s standsfor specific.
(11:06):
And what that means is we need to be veryspecific about what we're trying to measure,
what is the goal.
I recently actually wrote a post about how thisincreased KOL awareness has become a go to
metric for medical affairs teams, but it's notreally a good goal.
(11:26):
It's not a good good metric because it's justtoo broad to be measured.
Instead of looking at, these KOL awarenessmetrics, you can be looking at things like
message comprehension.
So how many KOLs walked away with the keymessages that you try to communicate to them,
scoring how well they understand yourscientific value proposition.
(11:49):
You can look at behavioral indicatorindicators.
So how many KOLs or what percentage of KOLs inyour target network is proactively reaching out
to get more information or, how frequently theyinitiate contact versus you trying to set up
meetings and follow-up with them.
The impact measurement could be another goodmetric, which is basically understanding the
(12:12):
scientific share of voice, the network effect.
So there are a lot of metrics that you canchoose from that are not just focused on
awareness and that can give you a lot moreclarity.
M stands for measurable.
So once you have the goal, you kind of have toask yourself, how would we know that we are
making progress on this goal?
(12:34):
And in order to be able to do that, you need tohave a baseline.
So establish your baseline and then decide,what would tell you that you're making progress
on this specific goal.
A, achievable.
For a nonpromotional function, this isextremely important.
Obviously, you want to be compliant, and youalso want to be realistic and appropriate.
(12:58):
So can we truly achieve this goal, or are webeing either too easy on ourselves or, you
know, we're aiming too high and we'll neverachieve that?
R is relevant, so tying the goal to a criticalbusiness outcome.
For example, is this message critical fordifferentiating in a crowded space?
(13:21):
Does it directly support an upcoming launch?
And if that metric doesn't or those metricsdon't, then it's great to have them.
They may look interesting, but they reallydon't accomplish anything.
And then finally, the most important one, theyneed to be time bound.
A lot of times, we kind of say, increaseawareness of this, increase awareness of that,
but there is no timeline attached to that.
(13:43):
So you need to know if you're trying to do thiswithin six months, within a year, within three
months, whatever that goal is.
So at the end of the day, it's it like I saidearlier, it forces you to be a lot more clear
and to look at observable behavior from HCPs asopposed to just tossing out some information
(14:05):
and hoping it sticks.
Got you.
Okay.
So let me unpack that for a second.
So SMART framework is specific, measurable,achievable, relevant, and time bound.
Okay.
Right?
We got that right?
Yes.
You got it.
For those of you taking notes, just wanna makesure.
(14:27):
And then so I wanna talk about and I'm gonnathrow some buzzwords out because I took notes
when we talked.
But I want you to talk about the concept ofscientific ROI
Mhmm.
And how share of scientific voice, which is aterm I learned from you, provides, a more
(14:47):
meaningful metric, than maybe some of thetraditional MSL activity metrics.
Right.
Yeah.
Scientific share of voice generally, share ofvoice has been a popular metric across all
teams for as long as I have been in this field.
(15:08):
But the scientific share of voice essentiallyasks a fundamental question of, are we
advancing the scientific conversation in a waythat reflects our strategy and highlights what
makes our science different?
The important thing when you're looking at thismetric of scientific share of voice is is it is
(15:31):
a lot less about teams medical affairs teamstalking and a lot more about them listening and
looking at whether their data shows up incritically important sources.
So that could be peer reviewed journals,independent presentations in congresses.
So you're you're trying to see if it's not justyour team that's communicating the message, but
(15:56):
it's else it's actually the field that isdiscussing it, citing it, or engaging with it.
And scientific share of voice is one of thebest ways to do that.
So instead of, you know, counting how manymeetings did we have, how many times did we met
meet with this HCP, you are actually asking thequestion of, is our signs showing up in the
(16:20):
places that matter?
Mhmm.
So to give you an example, let's say you'reworking on a novel approach in leukemia.
If your target mechanism or the, you know,mechanism of action process, whatever it is
that's critically important, starts showsstarts showing up in these peer reviewed
journals or presentations or, you know,wherever your HCP is appearing, then you know
(16:46):
that your data is and and you know that yourdata is contributing to that growth.
Now that's really impact, and that's somethingthat you can share and take to your leadership.
So then can you talk about how to track this
Yeah.
Concept?
Yeah.
It's, I think it a lot of people feel that it'svery intimidating, but it doesn't have to be.
(17:12):
It it really the easiest way the easiest placeto start is within your own organization.
Mhmm.
You can talk with your competitive intelligenceteams or publication planning teams.
They may have things in place already.
You can also work with external vendors.
(17:34):
Obviously, they do this kind of bibliometricanalysis if you have a budget for it.
And then finally, the simplest way to do it isthis manual review of literature that you can
use PubMed for as a starting point.
Let's say you do it every quarter.
You set up your keywords.
You set up your key messages.
(17:55):
Ideally, you tag them to your brand, yourcompetitor brands.
And then you will inevitably start seeing overtime how your share of voice is tracking
compared to that of your competitors.
So several ways to do it, and none of them arevery difficult.
Yeah.
I was expecting a more complicated answer.
(18:15):
Like, it was gonna be some like, but it doesn'tsound like that.
It sounds like there's tools and there'sthere's the ability to do this fairly simply.
Yeah.
It's it's all about, what kind of an appetiteyou have for it and what your budget is.
But you really there are several ways toapproach it just like you said.
(18:35):
It really kinda pick your poison.
Yeah.
Just do something.
Let's talk about the idea of message alignmentROI, which sounds very powerful to me.
How do you evaluate whether KOLs are trulyinternalizing and relaying that scientific your
(18:58):
scientific messages.
We talk about that concept?
Yeah.
Absolutely.
And message alignment, again, is another metricthat is widely used with teams that are, as I
mentioned earlier in the podcast, not revenuefocused.
Mhmm.
In general, this entire smart framework reallyfocuses on listening rather than talking.
(19:24):
Mhmm.
And for message alignment, you really want toit goes into whether the KOLs are taking your
scientific narrative and actually relayingthem.
So after your scientific engagements, whetherit's an ad board or a meeting at a congress or
(19:44):
a one on one meeting, you really want to beable to measure and look at whether the KOLs
are repeating your messages in their own words,in their own channels.
It could be the conference slides.
It could be their articles.
It could be even their social media posts,which is becoming increasingly more important.
(20:06):
Mhmm.
If they're a guest on a if they're a guest on apodcast, that could show up there.
So, you know, various channels to see if yourmessages are showing up.
Now, historically, this has been one of themost difficult things to measure because it was
very labor intensive.
Mhmm.
You had to have people kind of look set upkeywords and then manually compare those
(20:30):
keywords to the content and see what percentageis showing up.
But AI has really transformed it, made it very,very, possible and feasible, and you can do it
at scale.
If it is well trained, to recognize yourkeywords and your key messages, it actually
does an incredible job of capturing kind of howwell these are resonating.
(20:56):
So you can also do it through message audits ifyou don't have access to these tools,
analyzing, again, whether KOLs are referencing,you you set up keywords, you set up alerts.
And if you're they're not, then that's afeedback loop.
You go back to the drawing board, and you kindof try to understand, okay.
What is it that didn't resonate?
(21:18):
Where do I need to bring more clarity?
Am I meeting with the right person or people?
So all of those are critical to understandingwhether the messages are resonating with your
target audience or not.
Got you.
Okay.
So let's let's bring this back.
What would you say is the the proper balancebetween your, the smart outcome based metrics
(21:46):
and traditional activity metrics?
Is is like, should one outweigh the other?
I wouldn't necessarily say that one has tooutweigh the other.
If it is tied to your strategy, then it will itshould create a balance between the two because
I definitely don't want to dismiss theimportance of activity metrics.
(22:08):
Right?
They give you this operational insight, helpwith resource planning, managing the team comp
capacity, compliance reporting, which isincredibly important.
So you still need to know how many engagementsyou're having and with whom and how frequently.
But the outcome based metrics give youstrategic insight and answer those critically
(22:34):
important questions of are we advancing ourscientific narrative?
Are we differentiating ourselves among ourcompetitors?
I mean, I can give you an example.
If I told you the team met with a 100 KOLs,then that would sound impressive, but I don't
know if it's necessarily if it tells youanything other than they met with a 100 HCPs.
(22:59):
But if I told you 50% or 10% or 3% of thoseKOLs or HCPs walked away with a clear
understanding of our message, of our criticaldata points, and more importantly, they are
able to share it with their own network, Ithink that's where you can really tell the
(23:20):
difference and, you know, the combination ofthe two can be very powerful.
Right.
So let's let's so as I feel like as we'retalking to medical affairs leaders that may be
looking at some of these ideas and concepts toimplement, What how can they create a and build
(23:41):
a compelling business case for implementingsmart measurements to gain leadership buy in
for adopting this sort of system.
Yeah.
And and, again, it's a common problem.
Right?
Because it's not easy to if you again, youdon't have that fancy dashboard that you can
(24:05):
just walk in and show to your leadership.
So the best way to do it is start speaking theleadership language.
And as we know, regardless of the function, theleadership folk is what they care about the
most are risk mitigation, efficiency, andstrategic impact.
(24:25):
So if you show how these smart metrics help themedical affairs teams have impact, then it's
easier to get the leadership buy in.
The the easiest way to do it is start simple.
I always say don't don't try to boil the ocean.
It's really not critical.
What you want to do is pick one goal, pick onemessage, pick one, important area, and just
(24:53):
start with that.
And then if you build the smart frameworkaround that one key area, and you see results,
then take that to your leadership and kind ofexplain how that worked, where that worked, if
that didn't work, why it didn't work, and thenbuild from there.
I think a lot of teams and and this isprobably, something that needs to be addressed
(25:16):
by leadership teams is that they should givetheir teams the ability to experiment, to fail
even at times because you really can't learn ifyou're not failing.
Mhmm.
So having that leadership support is equallyimportant.
It's not just the team.
It's not just on the teams to do this right.
It's also on the leadership to encourage it andhelp kind of elevate this medical affairs
(25:41):
function from a support function to a strategicpartner.
Right.
So what about like, what advice would you givemedical affairs professionals at smaller
companies or companies that are, like, earlierstage?
Maybe they don't have a lot of data.
Yeah.
It would probably be a similar advice.
(26:02):
Start simple.
K.
You absolutely do not need a big budget, thefanciest platform, the most advanced AI
capability to start measuring.
Start with a basic Excel spreadsheet, PubMedalerts.
Maybe throw in some chat GPT if you must.
(26:22):
But, really, again, pick one priority message,one scientific narrative that's really
important for that early stage product, andthen start tracking where and how it shows up,
if it shows up, whether it's in literature, KOLdiscussions, social media posts.
And if it's not, how did you get there, developthat framework?
(26:43):
You can also use the the surveys, which we knoweven smaller companies have the post engagement
surveys.
So include a question.
What are the key two key takeaways that youtook from this meeting?
And then see if your, messages are showing up.
I think smaller companies have the advantage ofbeing, agile.
(27:06):
They're not bogged down by the big pharmabureaucracy.
So I would say use that nimbleness to youradvantage and build from there.
And how does all this differ from like, whatare the what about companies that are out there
and just like, well, we do insights gathering,and we just track everything through insights
and just like, do you run into that a lot?
(27:26):
Is that something that is a common notionamongst medical affairs teams?
Oh, yeah.
Oh, absolutely.
Every team thinks they have great insights.
But then when you try to actually measureimpact using those insights, it's very
difficult.
And I think a yeah.
They're not there because insights is such abroad term that again, a lot of people, teams
(27:52):
think that activity means insight, but itdoesn't.
Insight needs to deliver information that canbe used for further analysis.
If you just say, I met with this person, and Ishared this information with them, that can be
used.
There there's nothing in there that can leadto, an actionable I mean, it's an overused
(28:14):
term, but, it needs to be actionable.
You need to be able to do something with thatinsight.
Yeah.
So yeah.
A lot of teams have a lot of insights, but verylittle that they can work with to impact the
strategy.
Yeah.
So let's just say alright.
So the folks that are listening right now,let's say someone says, alright.
(28:34):
I'm in.
You got me.
Lucine, you you you used your your Russian spytactics and convinced me that I need to change
my metrics?
Like, where do they start?
What's one piece one thing that they can dothis week that can point them in this
direction?
Yeah.
I would say pick one scientific message and askyourself, how will we know if this message is
(29:02):
being understood and adopted by the people thatwe're trying to reach?
So that's the easiest way to start.
How will we know that we're making progress?
Then use the smart framework that we talkedabout a little earlier in the podcast to break
down that understanding, the indicators, thekey indicators along the way that will tell you
(29:25):
whether this message is being understood and,more importantly, disseminated by your key HCPs
or your KOLs.
Even just honestly, even just starting thatconversation changes the mar mindset.
It is less about building a perfect frameworkand following everything to the tee and more
(29:51):
about getting into the habit of thinkingstrategically.
I used to tell this to my you mentioned theRussian spy, but I used to tell my Pentagon
clients that the data doesn't need to beperfect.
The data doesn't need to be super clean.
If you do this consistently and over time, youwill see trends.
(30:11):
I promise you that.
There is just no way that trends are not goingto appear.
So start somewhere, keep it consistent, do itover time, and then learn from it.
Because it's a critical feedback loop whetheryou're successful or not.
It's something that forces you to dig deeperand truly understand whether you're making an
(30:33):
impact.
And if you are not, then what you need to do tohave that kind of an impact.
Yep.
That's that's that's, like, great life advicetoo.
Start somewhere, be consistent.
If you could do those two things, you'll besuccessful at least in some capacity.
Well, Lucine, thank you for for sharing thisinteresting this is like a new take on this
(30:57):
conversation.
So I appreciate you sharing these insights withus.
Thank you so much time Tom.
Sorry.
It it was great.
I really enjoyed speaking with you.
And, yeah, it's my favorite subject.
So
Yeah.
No.
You could see you're passionate about it.
Awesome.
Yeah.
Well, thank you.
And, guys, thank you all for listening.
Thank you for your support of this show.
If you got value in this, please share it withothers, And don't forget to, engage on
(31:22):
LinkedIn.
So if you see the announcements out there andyou wanna make a comment or or message me on
something, I always love to hear from you guys.
See you next time, guys.
Thanks.
Thank you, Tom.
Hey, guys.
Welcome to the podcast.
I am very excited to have another, valuediscussion.
I know that this is such an important topic forMSLs and for medical affairs.
And I have Lucine with me.
Lucine, welcome to the podcast.
Are you excited?
Very.
(00:22):
Thank you so much for having me.
I'm gonna let you let's guys.
So the title of this episode is frominteractions to impact, how medical affairs can
prove its value.
But, Lucine, why don't you do an introductionfirst so that you could tell everybody who you
are, and then we can we can just kinda jumpright into it.
(00:44):
Okay.
Thank you again, Tom.
I am really excited to be here.
I'm the founder of Cypherio, an AI enabledplatform that helps medical affairs teams
measure their true impact.
My entire background is actually in measurementand communications and analytics.
I've spent decades helping corporatecommunications teams at some very large
(01:05):
organizations measure the impact of their work.
And by very large, I that includes thePentagon.
I've actually spent a few years helpingPentagon measure their impact of
communications, and it's actually aninteresting story because it led to, as you can
(01:26):
imagine, several very interesting projects andseveral very interesting rumors.
When we moved to California, my daughter'sfriends were convinced that I'm a Russian spy.
I didn't exactly deny it, but I found out aboutit from one of the parents at a drop off.
So
maybe quite interesting.
(01:46):
Funny.
You I could see that.
I mean, you look the part.
You sound the part.
You get the whole thing down.
I'm going with that.
I'm gonna say I had a Russian spy on thepodcast.
Well, I'm I'm Armenian, so I don't even knowwhere that came from.
But somehow, somewhere, they they must havemade that connection.
Yep.
But in all seriousness, I've spent much of mycareer helping teams move beyond vanity metrics
(02:09):
and to measure their real impact, and that'swhat we're bringing to medical affairs and
hoping to discuss here at this podcast.
Well, the the one thing that I I think thereason I'm excited about this conversation, the
way it's gonna be a little bit different is weare gonna talk about impact.
We're gonna talk about metrics numb notmetrics, but analytics and numbers and not, you
(02:33):
know, measurement, not just concept.
So but before we get into that, I do have aquick announcement.
I wanna remind everybody to register for FiercePharma Week.
It's coming up very quickly, September, andthat's in Philadelphia at the Convention
Center.
This used to be Mass East.
(02:54):
It's called Fierce Pharma week now.
It's a much larger event because while it stillhas medical affairs, it also has PR and
communications, business development andlicensing, and pharma marketing.
So there's there's different functional areas,which is kinda cool because it's a bigger event
so you get to meet a lot of different people,but medical affairs is still its own its own
(03:19):
thing.
So for more information, go to Fierce PharmaWeek.
Just Google it.
When you register, use the code MSLTALK, nospaces, to get 25% off.
And I look forward to seeing you out therebecause I'm gonna definitely be there.
So, definitely register.
So, Lucine, let's get into this.
(03:41):
Why do you believe medical affairs, hashistorically struggled to quantify its value,
obviously, compared to commercial teams?
Right.
So, this is a problem that hits, severaldifferent depart departments, and I just
mentioned corporate communications a minuteago.
(04:02):
But the essential question is, how do youmeasure the value of roles that don't bring in
revenue?
As we know, from just our experience of workingwith medical affairs teams, medical affairs was
not originally built for ROI tracking.
It kind of evolved from that support functionfocused on scientific integrity.
(04:25):
The impact of medical affairs, the way we seeit, is often long term and indirect, which
makes it very difficult to measure usingtraditional business metrics.
Commercial teams have long had theinfrastructure and this data driven mindset.
Medical affairs have also operated with adifferent purpose, the commercial than
(04:48):
commercial.
Right?
We, with medical affairs, we focus on advancingthe scientific narrative, the scientific
integrity, whereas commercial teams are builtall around revenue and quotas and measurable
outcomes.
So to give an example, medical affairs canspend many, many hours.
(05:12):
The teams can spend many hours meeting withKOLs, and, eventually, the result may, be the
inclusion in guidelines, but that may take acouple of years.
That's not exactly something that shows up on apretty dashboard.
Whereas commercial teams can very clearly say,we're in this campaign, and we saw this many
(05:33):
and we saw an increase in prescriptions or, youknow, this percentage change.
So it's it's a little it's much more clear cutand easier to measure than what we see for
medical affairs.
The what's tricky about the work of medicalaffairs also, it's it's very subtle.
You're helping shape the scientificconversation, the scientific narrative,
(05:57):
supporting evidence generation.
But, again, as I mentioned earlier, that's notsomething that you can easily capture in a
dashboard.
Yeah.
Well and it's just I mean, it's it's so easyfor commercial.
There's like you said, there's just so manyways to track it.
So how do you it's so hard to track it in themedical affairs world.
(06:19):
It's it's not as as easily cut and dry becauseyou can't just look at sales and say, okay.
Well, we did this, and then that influenced thenumber of sales that we had.
And there's the return on our investment andour time.
So let's talk about when we talk about, some ofthe ways that MSLs are being tracked, like
(06:42):
metrics, for example, would you say there's anyunintended consequences of relying too heavily
on what we see is a lot of activity basedmetrics, like number of interactions.
Can we talk about that?
Yeah.
Absolutely.
And on the surface, that looks great.
Right?
You capture how many meetings you had, howfrequently you meet with your KOLs in your
(07:07):
territory or HCPs in your territory.
So, it's a great metric, again, on paper and ondashboards.
But in essence, when you are only measuringthese activity metrics, you are incentivizing
volume.
So instead of doing better, you are focusing onjust doing more.
(07:28):
And this can lead to this false sense ofefficiency that, oh, we've had this many
meetings, so we're doing great.
But what if those meetings did not have theimpact, the intended impact that you're trying
to achieve?
For example, let's say an MSL meet met with 50HCPs in a month or a 100 HCPs, and that's what
(07:51):
we see more often than not.
But, again, what does that mean?
Did these meetings lead to any meaningfulchange?
Did the HCPs walk away with something that youcan actually follow-up with and see whether it
had the impact you intended?
Where did they understand the key messages, thekey data points that you were sharing?
(08:14):
You don't know.
Did you meet with the right people?
Essentially, you're busy, but you're notnecessarily sure if you're effective or not.
The activity metrics also have a tendency tocreate this illusion of productivity.
You show high numbers to leadership.
They feel like the goals are accomplished.
(08:34):
But in reality, as I said, it just creates itsfull sense of effectiveness and accomplishment
when in reality, you may not have accomplishedthat.
Now with that being said, there are certainterritories that are it depends.
Right?
If you're in a smaller territory and you'reearlier in the product life cycle, it makes
more sense to focus on the number of meetingsbecause you may not have access to top KOLs or
(09:00):
top institutions.
And so, therefore, the the frequency ofmeeting, the frequency of these touch points,
the length of those meetings matters becauseyou're trying to introduce a new concept.
You're building awareness around the disease.
And so the more meetings you have, the morebeneficial it is for your team.
(09:20):
So at the end of the day, I would say thebalance is determined by your strategy.
If you tie your metrics to your strategy, thenthe decision of whether you're focusing on
quantitative or qualitative or what the rightcombination of the two is, it becomes a lot
more obvious.
Well, let's I wanna talk about like, let'slet's get into that piece of it because I know
(09:44):
when you and I spoke Mhmm.
Originally, you had mentioned the SMARTframework.
So you have this SMART framework.
Mhmm.
Can you break that down for me?
Like, so how can medical affairs apply thecomponents of what your SMART framework is, and
how can that help them meet their goals?
(10:05):
Yeah.
Absolutely.
So the SMART framework is what the corporatecommunications teams adopted a while ago, I
would say about fifteen years ago.
And I don't mean all of the teams, but the onesthat wanted to show real impact.
Because, initially, all these teams weresimilar to medical affairs.
(10:25):
We're measuring media impressions, numb thenumber of whips website clicks, number of
visitors, how many press releases they issued.
And while that looked great on paper, again,were all vanity metrics that didn't really show
if they were accomplishing the intended purposeand if they were having the right impact.
(10:45):
So they adopted what they call the smartframework, measurement framework.
And the biggest advantage of this smartframework is that it forces you to be very
clear and to break down the goalsystematically.
Essentially, if we walk through it, s standsfor specific.
(11:06):
And what that means is we need to be veryspecific about what we're trying to measure,
what is the goal.
I recently actually wrote a post about how thisincreased KOL awareness has become a go to
metric for medical affairs teams, but it's notreally a good goal.
(11:26):
It's not a good good metric because it's justtoo broad to be measured.
Instead of looking at, these KOL awarenessmetrics, you can be looking at things like
message comprehension.
So how many KOLs walked away with the keymessages that you try to communicate to them,
scoring how well they understand yourscientific value proposition.
(11:49):
You can look at behavioral indicatorindicators.
So how many KOLs or what percentage of KOLs inyour target network is proactively reaching out
to get more information or, how frequently theyinitiate contact versus you trying to set up
meetings and follow-up with them.
The impact measurement could be another goodmetric, which is basically understanding the
(12:12):
scientific share of voice, the network effect.
So there are a lot of metrics that you canchoose from that are not just focused on
awareness and that can give you a lot moreclarity.
M stands for measurable.
So once you have the goal, you kind of have toask yourself, how would we know that we are
making progress on this goal?
(12:34):
And in order to be able to do that, you need tohave a baseline.
So establish your baseline and then decide,what would tell you that you're making progress
on this specific goal.
A, achievable.
For a nonpromotional function, this isextremely important.
Obviously, you want to be compliant, and youalso want to be realistic and appropriate.
(12:58):
So can we truly achieve this goal, or are webeing either too easy on ourselves or, you
know, we're aiming too high and we'll neverachieve that?
R is relevant, so tying the goal to a criticalbusiness outcome.
For example, is this message critical fordifferentiating in a crowded space?
(13:21):
Does it directly support an upcoming launch?
And if that metric doesn't or those metricsdon't, then it's great to have them.
They may look interesting, but they reallydon't accomplish anything.
And then finally, the most important one, theyneed to be time bound.
A lot of times, we kind of say, increaseawareness of this, increase awareness of that,
but there is no timeline attached to that.
(13:43):
So you need to know if you're trying to do thiswithin six months, within a year, within three
months, whatever that goal is.
So at the end of the day, it's it like I saidearlier, it forces you to be a lot more clear
and to look at observable behavior from HCPs asopposed to just tossing out some information
(14:05):
and hoping it sticks.
Got you.
Okay.
So let me unpack that for a second.
So SMART framework is specific, measurable,achievable, relevant, and time bound.
Okay.
Right?
We got that right?
Yes.
You got it.
For those of you taking notes, just wanna makesure.
(14:27):
And then so I wanna talk about and I'm gonnathrow some buzzwords out because I took notes
when we talked.
But I want you to talk about the concept ofscientific ROI
Mhmm.
And how share of scientific voice, which is aterm I learned from you, provides, a more
(14:47):
meaningful metric, than maybe some of thetraditional MSL activity metrics.
Right.
Yeah.
Scientific share of voice generally, share ofvoice has been a popular metric across all
teams for as long as I have been in this field.
(15:08):
But the scientific share of voice essentiallyasks a fundamental question of, are we
advancing the scientific conversation in a waythat reflects our strategy and highlights what
makes our science different?
The important thing when you're looking at thismetric of scientific share of voice is is it is
(15:31):
a lot less about teams medical affairs teamstalking and a lot more about them listening and
looking at whether their data shows up incritically important sources.
So that could be peer reviewed journals,independent presentations in congresses.
So you're you're trying to see if it's not justyour team that's communicating the message, but
(15:56):
it's else it's actually the field that isdiscussing it, citing it, or engaging with it.
And scientific share of voice is one of thebest ways to do that.
So instead of, you know, counting how manymeetings did we have, how many times did we met
meet with this HCP, you are actually asking thequestion of, is our signs showing up in the
(16:20):
places that matter?
Mhmm.
So to give you an example, let's say you'reworking on a novel approach in leukemia.
If your target mechanism or the, you know,mechanism of action process, whatever it is
that's critically important, starts showsstarts showing up in these peer reviewed
journals or presentations or, you know,wherever your HCP is appearing, then you know
(16:46):
that your data is and and you know that yourdata is contributing to that growth.
Now that's really impact, and that's somethingthat you can share and take to your leadership.
So then can you talk about how to track this
Yeah.
Concept?
Yeah.
It's, I think it a lot of people feel that it'svery intimidating, but it doesn't have to be.
(17:12):
It it really the easiest way the easiest placeto start is within your own organization.
Mhmm.
You can talk with your competitive intelligenceteams or publication planning teams.
They may have things in place already.
You can also work with external vendors.
(17:34):
Obviously, they do this kind of bibliometricanalysis if you have a budget for it.
And then finally, the simplest way to do it isthis manual review of literature that you can
use PubMed for as a starting point.
Let's say you do it every quarter.
You set up your keywords.
You set up your key messages.
(17:55):
Ideally, you tag them to your brand, yourcompetitor brands.
And then you will inevitably start seeing overtime how your share of voice is tracking
compared to that of your competitors.
So several ways to do it, and none of them arevery difficult.
Yeah.
I was expecting a more complicated answer.
(18:15):
Like, it was gonna be some like, but it doesn'tsound like that.
It sounds like there's tools and there'sthere's the ability to do this fairly simply.
Yeah.
It's it's all about, what kind of an appetiteyou have for it and what your budget is.
But you really there are several ways toapproach it just like you said.
(18:35):
It really kinda pick your poison.
Yeah.
Just do something.
Let's talk about the idea of message alignmentROI, which sounds very powerful to me.
How do you evaluate whether KOLs are trulyinternalizing and relaying that scientific your
(18:58):
scientific messages.
We talk about that concept?
Yeah.
Absolutely.
And message alignment, again, is another metricthat is widely used with teams that are, as I
mentioned earlier in the podcast, not revenuefocused.
Mhmm.
In general, this entire smart framework reallyfocuses on listening rather than talking.
(19:24):
Mhmm.
And for message alignment, you really want toit goes into whether the KOLs are taking your
scientific narrative and actually relayingthem.
So after your scientific engagements, whetherit's an ad board or a meeting at a congress or
(19:44):
a one on one meeting, you really want to beable to measure and look at whether the KOLs
are repeating your messages in their own words,in their own channels.
It could be the conference slides.
It could be their articles.
It could be even their social media posts,which is becoming increasingly more important.
(20:06):
Mhmm.
If they're a guest on a if they're a guest on apodcast, that could show up there.
So, you know, various channels to see if yourmessages are showing up.
Now, historically, this has been one of themost difficult things to measure because it was
very labor intensive.
Mhmm.
You had to have people kind of look set upkeywords and then manually compare those
(20:30):
keywords to the content and see what percentageis showing up.
But AI has really transformed it, made it very,very, possible and feasible, and you can do it
at scale.
If it is well trained, to recognize yourkeywords and your key messages, it actually
does an incredible job of capturing kind of howwell these are resonating.
(20:56):
So you can also do it through message audits ifyou don't have access to these tools,
analyzing, again, whether KOLs are referencing,you you set up keywords, you set up alerts.
And if you're they're not, then that's afeedback loop.
You go back to the drawing board, and you kindof try to understand, okay.
What is it that didn't resonate?
(21:18):
Where do I need to bring more clarity?
Am I meeting with the right person or people?
So all of those are critical to understandingwhether the messages are resonating with your
target audience or not.
Got you.
Okay.
So let's let's bring this back.
What would you say is the the proper balancebetween your, the smart outcome based metrics
(21:46):
and traditional activity metrics?
Is is like, should one outweigh the other?
I wouldn't necessarily say that one has tooutweigh the other.
If it is tied to your strategy, then it will itshould create a balance between the two because
I definitely don't want to dismiss theimportance of activity metrics.
(22:08):
Right?
They give you this operational insight, helpwith resource planning, managing the team comp
capacity, compliance reporting, which isincredibly important.
So you still need to know how many engagementsyou're having and with whom and how frequently.
But the outcome based metrics give youstrategic insight and answer those critically
(22:34):
important questions of are we advancing ourscientific narrative?
Are we differentiating ourselves among ourcompetitors?
I mean, I can give you an example.
If I told you the team met with a 100 KOLs,then that would sound impressive, but I don't
know if it's necessarily if it tells youanything other than they met with a 100 HCPs.
(22:59):
But if I told you 50% or 10% or 3% of thoseKOLs or HCPs walked away with a clear
understanding of our message, of our criticaldata points, and more importantly, they are
able to share it with their own network, Ithink that's where you can really tell the
(23:20):
difference and, you know, the combination ofthe two can be very powerful.
Right.
So let's let's so as I feel like as we'retalking to medical affairs leaders that may be
looking at some of these ideas and concepts toimplement, What how can they create a and build
(23:41):
a compelling business case for implementingsmart measurements to gain leadership buy in
for adopting this sort of system.
Yeah.
And and, again, it's a common problem.
Right?
Because it's not easy to if you again, youdon't have that fancy dashboard that you can
(24:05):
just walk in and show to your leadership.
So the best way to do it is start speaking theleadership language.
And as we know, regardless of the function, theleadership folk is what they care about the
most are risk mitigation, efficiency, andstrategic impact.
(24:25):
So if you show how these smart metrics help themedical affairs teams have impact, then it's
easier to get the leadership buy in.
The the easiest way to do it is start simple.
I always say don't don't try to boil the ocean.
It's really not critical.
What you want to do is pick one goal, pick onemessage, pick one, important area, and just
(24:53):
start with that.
And then if you build the smart frameworkaround that one key area, and you see results,
then take that to your leadership and kind ofexplain how that worked, where that worked, if
that didn't work, why it didn't work, and thenbuild from there.
I think a lot of teams and and this isprobably, something that needs to be addressed
(25:16):
by leadership teams is that they should givetheir teams the ability to experiment, to fail
even at times because you really can't learn ifyou're not failing.
Mhmm.
So having that leadership support is equallyimportant.
It's not just the team.
It's not just on the teams to do this right.
It's also on the leadership to encourage it andhelp kind of elevate this medical affairs
(25:41):
function from a support function to a strategicpartner.
Right.
So what about like, what advice would you givemedical affairs professionals at smaller
companies or companies that are, like, earlierstage?
Maybe they don't have a lot of data.
Yeah.
It would probably be a similar advice.
(26:02):
Start simple.
K.
You absolutely do not need a big budget, thefanciest platform, the most advanced AI
capability to start measuring.
Start with a basic Excel spreadsheet, PubMedalerts.
Maybe throw in some chat GPT if you must.
(26:22):
But, really, again, pick one priority message,one scientific narrative that's really
important for that early stage product, andthen start tracking where and how it shows up,
if it shows up, whether it's in literature, KOLdiscussions, social media posts.
And if it's not, how did you get there, developthat framework?
(26:43):
You can also use the the surveys, which we knoweven smaller companies have the post engagement
surveys.
So include a question.
What are the key two key takeaways that youtook from this meeting?
And then see if your, messages are showing up.
I think smaller companies have the advantage ofbeing, agile.
(27:06):
They're not bogged down by the big pharmabureaucracy.
So I would say use that nimbleness to youradvantage and build from there.
And how does all this differ from like, whatare the what about companies that are out there
and just like, well, we do insights gathering,and we just track everything through insights
and just like, do you run into that a lot?
(27:26):
Is that something that is a common notionamongst medical affairs teams?
Oh, yeah.
Oh, absolutely.
Every team thinks they have great insights.
But then when you try to actually measureimpact using those insights, it's very
difficult.
And I think a yeah.
They're not there because insights is such abroad term that again, a lot of people, teams
(27:52):
think that activity means insight, but itdoesn't.
Insight needs to deliver information that canbe used for further analysis.
If you just say, I met with this person, and Ishared this information with them, that can be
used.
There there's nothing in there that can leadto, an actionable I mean, it's an overused
(28:14):
term, but, it needs to be actionable.
You need to be able to do something with thatinsight.
Yeah.
So yeah.
A lot of teams have a lot of insights, but verylittle that they can work with to impact the
strategy.
Yeah.
So let's just say alright.
So the folks that are listening right now,let's say someone says, alright.
(28:34):
I'm in.
You got me.
Lucine, you you you used your your Russian spytactics and convinced me that I need to change
my metrics?
Like, where do they start?
What's one piece one thing that they can dothis week that can point them in this
direction?
Yeah.
I would say pick one scientific message and askyourself, how will we know if this message is
(29:02):
being understood and adopted by the people thatwe're trying to reach?
So that's the easiest way to start.
How will we know that we're making progress?
Then use the smart framework that we talkedabout a little earlier in the podcast to break
down that understanding, the indicators, thekey indicators along the way that will tell you
(29:25):
whether this message is being understood and,more importantly, disseminated by your key HCPs
or your KOLs.
Even just honestly, even just starting thatconversation changes the mar mindset.
It is less about building a perfect frameworkand following everything to the tee and more
(29:51):
about getting into the habit of thinkingstrategically.
I used to tell this to my you mentioned theRussian spy, but I used to tell my Pentagon
clients that the data doesn't need to beperfect.
The data doesn't need to be super clean.
If you do this consistently and over time, youwill see trends.
(30:11):
I promise you that.
There is just no way that trends are not goingto appear.
So start somewhere, keep it consistent, do itover time, and then learn from it.
Because it's a critical feedback loop whetheryou're successful or not.
It's something that forces you to dig deeperand truly understand whether you're making an
(30:33):
impact.
And if you are not, then what you need to do tohave that kind of an impact.
Yep.
That's that's that's, like, great life advicetoo.
Start somewhere, be consistent.
If you could do those two things, you'll besuccessful at least in some capacity.
Well, Lucine, thank you for for sharing thisinteresting this is like a new take on this
(30:57):
conversation.
So I appreciate you sharing these insights withus.
Thank you so much time Tom.
Sorry.
It it was great.
I really enjoyed speaking with you.
And, yeah, it's my favorite subject.
So
Yeah.
No.
You could see you're passionate about it.
Awesome.
Yeah.
Well, thank you.
And, guys, thank you all for listening.
Thank you for your support of this show.
If you got value in this, please share it withothers, And don't forget to, engage on
(31:22):
LinkedIn.
So if you see the announcements out there andyou wanna make a comment or or message me on
something, I always love to hear from you guys.
See you next time, guys.
Thanks.
Thank you, Tom.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
New Heights with Jason & Travis Kelce
Football’s funniest family duo — Jason Kelce of the Philadelphia Eagles and Travis Kelce of the Kansas City Chiefs — team up to provide next-level access to life in the league as it unfolds. The two brothers and Super Bowl champions drop weekly insights about the weekly slate of games and share their INSIDE perspectives on trending NFL news and sports headlines. They also endlessly rag on each other as brothers do, chat the latest in pop culture and welcome some very popular and well-known friends to chat with them. Check out new episodes every Wednesday. Follow New Heights on the Wondery App, YouTube or wherever you get your podcasts. You can listen to new episodes early and ad-free, and get exclusive content on Wondery+. Join Wondery+ in the Wondery App, Apple Podcasts or Spotify. And join our new membership for a unique fan experience by going to the New Heights YouTube channel now!