December 7, 2023 • 15 mins
An interview on CGM accuracy and standardization with Guido Freckmann, MD, Medical Director of the Institute for Diabetes Technology Research at Ulm University in Germany.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
David Klonoff (00:15):
Welcome to Diabetes Technology Report.
This is the podcast devoted todiabetes technology.
I'm Dr David Klonoff.
I'm an endocrinologist at MillsPeninsula Medical Center in San
Mateo in UCSF.
I'm going to introduce ourother co-host, Dr David Kerr,
who will introduce our guesttoday.
David Kerr (00:35):
Thanks, david, and hello to everyone listening
today.
I'm David Kerr.
I'm a diabetes researcher andbased in Santa Barbara,
california.
Today's very special guest isGuido Freckman, who many of you
know is really a world authorityon glucose monitoring in all of
its shapes and forms.
Welcome, guido.
(00:56):
It's great to see you and hearfrom you, rather Just to begin
for our listeners to really toset the scene.
How did you get interested indiabetes technology, glucose
monitoring?
Where did that come from?
Guido Freckmann (01:13):
Yes, David, Thank you very much for your
kind introduction.
To introduce myself, I'm GuidoFreckman from the Institute of
Diabetes Technology in OldGermany.
It was a bit accidental that Icame to glucose monitoring.
I came to this institute in1999, starting with research on
(01:36):
algorithms Today you would sayfor AID systems.
From these studies we gathereda lot of BGM data.
After a couple of years ofthese studies, we started with
the evaluation of BGM data andthen recognized that there
(01:58):
became a standard available.
We started performing studiesaccording to the standard.
David Kerr (02:07):
What are you working on at the moment in this area?
What's your passion today?
Guido Freckmann (02:13):
Yes, after working more than 10 years, we
did lots of evaluation studieswith BGM.
In parallel we did many studiesfor the development of CGM or
for CGMs under developmentCurrently.
We performed both.
(02:36):
In the last years we tested acouple of CGMs which are
currently under development.
We also performed BGM studies.
Since 1999, I'm chairing aworking group on CGM to bring
something that's alreadyavailable in BGM an
international standard also tothe CGM scene.
(03:00):
I'm very proud that DavidKlonoff is also one of the
members of this working group.
We are looking on studyprocedures on metrics and under
gaps.
We are also looking on studyprocedures on metrics and under
gaps that's left by the currentguidelines and performed some
(03:25):
publications and some work inthe last years.
Our goal is to support thedevelopment of an international
standard to make CGM studiesmore comparable.
David Klonoff (03:42):
Guido, why do you feel that it's important to
make CGM standards comparable?
Guido Freckmann (03:49):
Yeah, I think there are currently some really
good CGMs available, but thewhole study procedures are not
defined, that it's difficult tocompare the results of different
studies and especially now wesee, especially here in Europe,
many devices coming from Asianow and most of them are
(04:14):
proposing a very good mark.
It's very difficult todistinguish.
Is this really the accuracy youwill have in daily life, or
what is better, what is worse?
And it would be very helpful ifwe have a standardized
procedure that you can bettercompare different devices and
see is it good for adjunctive ornon-adjunctive use.
David Klonoff (04:39):
Guido, do you think that MARD is the best
metric for defining theperformance of a CGM, and could
you also comment on the metricthat you developed, cgdiva?
Yeah, so.
Guido Freckmann (04:54):
I would say MARD is the most used metric for
CGM because it's only oneparameter and it's easy to show
this.
David Kerr (05:06):
Just for our listeners.
Can you explain, mard, just sothat everyone's clear what that
means?
Guido Freckmann (05:12):
So the MARD is a mean, relative, absolute
difference and it's a comparisonbetween the CGM data and the
comparator data, which istypically a BGM or a BGM lab
device, and you take the averageof all the differences you
(05:33):
measure in the study and it'sreally very dependent on study
design, on frequency ofmeasurements and so on, and so
it's averaging a lot and notreally showing the difference in
the details.
And that was a reason for us todevelop the CGDiva, which is
(05:55):
based on the ICGM criteria ofthe FDA and we try to visualize
the data, to show the median andthe distribution and to give,
on a visual base, moreinformation than the MARD has or
the MARD provides Guido.
David Klonoff (06:15):
one more question if a device has a high CGDiva,
meaning that there's a widedistribution, how would you use
a device like that differentlythan another device which has a
narrow distribution?
Guido Freckmann (06:31):
Yeah, so I would first look.
We have proposed two plots, soan average plot including all
the data and then a plot withthe single devices.
And I would look how are thesingle devices are working?
Is there a high distributionbetween the devices?
Is there a high differencebetween single devices?
(06:54):
And if the or is there asystematic bias which is
reproducible and you can correctfor?
Or is there a high noise?
So, looking on all this data,you can distinguish between the
devices and say, okay, it's goodfor this or probably not so
(07:14):
good for that goal.
David Kerr (07:17):
Guido, just taking that rather technical
description there, looking at itfrom a point of view of people
with diabetes.
Why is this important and doesit matter in different
situations?
This accuracy, why does itreally matter?
Guido Freckmann (07:33):
no-transcript.
I think it matters, especiallyif the and that's another
proposal we made for the studyprocedures if there is a higher
rate of change and you should beone before a hypo and if you
have a long time delay, thewarning is probably late.
And these are situations wherethe study procedure is important
(07:59):
.
And for the MART in the CGDVyou can see for the lower values
and the higher values is therethe same bias or is there
probably a different bias thathypo's are over or
underestimated and that canreally have impact for the
(08:19):
diabetic and for the hypowarning and other things in his
daily life.
David Kerr (08:27):
So this is really important and probably it's
going to affect the decisionmaking about which technology is
the most appropriate for peoplewith diabetes.
That's where this is going.
David Klonoff (08:39):
Guido, what would you say is the effect of safety
?
If you have either a high MARTor if you have a wide CGDV, will
this affect safety?
Guido Freckmann (08:54):
So it's very interesting to look especially
on the single sensors.
If you have a high distributionof all the sensors, I think
this affects safety.
If most of the sensors arelooking good but only one or two
are deviating, it could affectsafety for this one or two and
it's important to detect them.
And yeah, I think it depends onthe degree of differences in
(09:22):
this plot if it affects safetyor not, but it can affect.
David Klonoff (09:28):
Are you seeing products coming onto the market
in Europe where there's nopublished data about their
accuracy?
Guido Freckmann (09:36):
Yeah, most of the products coming to the
market have their own data andthe question is how this data
are achieved.
Is there how many rates ofchange, distribution of values
and so on?
So we tested last year a devicethat proposed to have an MRD
(09:57):
below 10 and with a factorycalibrated mode.
It wasn't about double the MRDand with one day calibration it
was still 15.
So I think it's very importantto have comparable procedures to
really see to get comparablevalues, output values out.
David Kerr (10:20):
Yeah, so it seems that's very reassuring for
people with diabetes andclinicians that this accuracy
question needs to be constantlythought about.
I'm sure many of our listenerswould like to know where do you
see non-invasive devices at themoment?
Where are they at the moment interms of accuracy?
(10:40):
Is this really going to happensoon or is this still a pie in
the sky?
Guido Freckmann (10:50):
So my insight into non-invasive devices is
limited, but I know some deviceswhich can already measure I'm
not sure in all devices aroundthere and, interestingly, we
bought two watches who said theycan measure sugar from Asia,
(11:14):
and my colleagues wear thesewatches two days and they show
on each day the same curve.
And on the third day we gave itto a banana and the banana
showed the same curve, like mycolleague.
So these watches are alreadyavailable.
We got them from 50 to 200euros and I think in future we
(11:42):
will have working non-invasivedevices.
But I think, as I know, thecurrent non-invasive devices are
there where CGM was about 10years ago or a little bit more,
and they are developing and theywill come, but I think we need
some more time.
David Klonoff (12:01):
Peter, what advice would you give to a
scientist who wants to develop anew glucose monitor?
Guido Freckmann (12:12):
Yeah, I think many people, especially in the
development of non-invasivedevices, are very optimistic and
they should be, in some case,realistic and looking.
What are the first steps?
You need to go step by step andyou will not have with the
first device, and that's thesame with CGM sensors.
(12:33):
You will need some years toimprove your algorithms and your
device to be in the statuswhere the current CGMs or VCGMs
are.
David Klonoff (12:46):
Is there anything else that you'd like to tell
the audience about yourexperiences or your
recommendations?
Guido Freckmann (12:55):
Yeah, currently .
I already told that wesubmitted a procedure.
We think is very important touse a comparable procedure for
the studies and we are currentlypreparing a study for this and
currently looking for funding,which is not very easy, and I
(13:15):
hope we can get the funding thatwe can perform a study with the
currently well-known devices toset something like a benchmark
with the study.
Where is it if we use moreparameters like CGDVA and not
only MRD, with a really definedstudy procedure?
David Kerr (13:36):
Just a philosophical question.
Is it ever going to be the casethat the difference is going to
be zero, or is it a?
What's the holy grail ofaccuracy for glucose monitoring?
Do we know what that is, or isit just pull it?
Guido Freckmann (13:53):
there.
That's a very good question.
If you measure capillary orvenous you have a difference and
we have seen in health C's inan OGTT post-prandial up to 30%
difference between these twocompartments.
So it's a bit difficult in thebody.
It depends on the place youmeasure.
(14:15):
You will have differences andtherefore the accuracy need to
be for a degree that is, I wouldsay, within 10 to 20% to make
no larger failures.
But it's very difficult to.
You can measure it in a labdevice to a high degree of
accuracy, but on the way to themeasurement are already
(14:38):
differences which you cannot getrid of.
David Klonoff (14:43):
Guido, thank you for speaking with us today.
You're definitely a worldleader in glucose monitoring and
I hope people listen to whatyou had to say so to the
audience.
Thank you for listening to thediabetes technology report.
We're available on Spotify andthe Apple store.
On behalf of David Kerr andmyself, Guido Fragman, thank you
(15:06):
and we will catch up with youat the next podcast.
Bye, everybody.
Thank you very much.
Welcome to Diabetes Technology Report.
This is the podcast devoted todiabetes technology.
I'm Dr David Klonoff.
I'm an endocrinologist at MillsPeninsula Medical Center in San
Mateo in UCSF.
I'm going to introduce ourother co-host, Dr David Kerr,
who will introduce our guesttoday.
David Kerr (00:35):
Thanks, david, and hello to everyone listening
today.
I'm David Kerr.
I'm a diabetes researcher andbased in Santa Barbara,
california.
Today's very special guest isGuido Freckman, who many of you
know is really a world authorityon glucose monitoring in all of
its shapes and forms.
Welcome, guido.
(00:56):
It's great to see you and hearfrom you, rather Just to begin
for our listeners to really toset the scene.
How did you get interested indiabetes technology, glucose
monitoring?
Where did that come from?
Guido Freckmann (01:13):
Yes, David, Thank you very much for your
kind introduction.
To introduce myself, I'm GuidoFreckman from the Institute of
Diabetes Technology in OldGermany.
It was a bit accidental that Icame to glucose monitoring.
I came to this institute in1999, starting with research on
(01:36):
algorithms Today you would sayfor AID systems.
From these studies we gathereda lot of BGM data.
After a couple of years ofthese studies, we started with
the evaluation of BGM data andthen recognized that there
(01:58):
became a standard available.
We started performing studiesaccording to the standard.
David Kerr (02:07):
What are you working on at the moment in this area?
What's your passion today?
Guido Freckmann (02:13):
Yes, after working more than 10 years, we
did lots of evaluation studieswith BGM.
In parallel we did many studiesfor the development of CGM or
for CGMs under developmentCurrently.
We performed both.
(02:36):
In the last years we tested acouple of CGMs which are
currently under development.
We also performed BGM studies.
Since 1999, I'm chairing aworking group on CGM to bring
something that's alreadyavailable in BGM an
international standard also tothe CGM scene.
(03:00):
I'm very proud that DavidKlonoff is also one of the
members of this working group.
We are looking on studyprocedures on metrics and under
gaps.
We are also looking on studyprocedures on metrics and under
gaps that's left by the currentguidelines and performed some
(03:25):
publications and some work inthe last years.
Our goal is to support thedevelopment of an international
standard to make CGM studiesmore comparable.
David Klonoff (03:42):
Guido, why do you feel that it's important to
make CGM standards comparable?
Guido Freckmann (03:49):
Yeah, I think there are currently some really
good CGMs available, but thewhole study procedures are not
defined, that it's difficult tocompare the results of different
studies and especially now wesee, especially here in Europe,
many devices coming from Asianow and most of them are
(04:14):
proposing a very good mark.
It's very difficult todistinguish.
Is this really the accuracy youwill have in daily life, or
what is better, what is worse?
And it would be very helpful ifwe have a standardized
procedure that you can bettercompare different devices and
see is it good for adjunctive ornon-adjunctive use.
David Klonoff (04:39):
Guido, do you think that MARD is the best
metric for defining theperformance of a CGM, and could
you also comment on the metricthat you developed, cgdiva?
Yeah, so.
Guido Freckmann (04:54):
I would say MARD is the most used metric for
CGM because it's only oneparameter and it's easy to show
this.
David Kerr (05:06):
Just for our listeners.
Can you explain, mard, just sothat everyone's clear what that
means?
Guido Freckmann (05:12):
So the MARD is a mean, relative, absolute
difference and it's a comparisonbetween the CGM data and the
comparator data, which istypically a BGM or a BGM lab
device, and you take the averageof all the differences you
(05:33):
measure in the study and it'sreally very dependent on study
design, on frequency ofmeasurements and so on, and so
it's averaging a lot and notreally showing the difference in
the details.
And that was a reason for us todevelop the CGDiva, which is
(05:55):
based on the ICGM criteria ofthe FDA and we try to visualize
the data, to show the median andthe distribution and to give,
on a visual base, moreinformation than the MARD has or
the MARD provides Guido.
David Klonoff (06:15):
one more question if a device has a high CGDiva,
meaning that there's a widedistribution, how would you use
a device like that differentlythan another device which has a
narrow distribution?
Guido Freckmann (06:31):
Yeah, so I would first look.
We have proposed two plots, soan average plot including all
the data and then a plot withthe single devices.
And I would look how are thesingle devices are working?
Is there a high distributionbetween the devices?
Is there a high differencebetween single devices?
(06:54):
And if the or is there asystematic bias which is
reproducible and you can correctfor?
Or is there a high noise?
So, looking on all this data,you can distinguish between the
devices and say, okay, it's goodfor this or probably not so
(07:14):
good for that goal.
David Kerr (07:17):
Guido, just taking that rather technical
description there, looking at itfrom a point of view of people
with diabetes.
Why is this important and doesit matter in different
situations?
This accuracy, why does itreally matter?
Guido Freckmann (07:33):
no-transcript.
I think it matters, especiallyif the and that's another
proposal we made for the studyprocedures if there is a higher
rate of change and you should beone before a hypo and if you
have a long time delay, thewarning is probably late.
And these are situations wherethe study procedure is important
(07:59):
.
And for the MART in the CGDVyou can see for the lower values
and the higher values is therethe same bias or is there
probably a different bias thathypo's are over or
underestimated and that canreally have impact for the
(08:19):
diabetic and for the hypowarning and other things in his
daily life.
David Kerr (08:27):
So this is really important and probably it's
going to affect the decisionmaking about which technology is
the most appropriate for peoplewith diabetes.
That's where this is going.
David Klonoff (08:39):
Guido, what would you say is the effect of safety
?
If you have either a high MARTor if you have a wide CGDV, will
this affect safety?
Guido Freckmann (08:54):
So it's very interesting to look especially
on the single sensors.
If you have a high distributionof all the sensors, I think
this affects safety.
If most of the sensors arelooking good but only one or two
are deviating, it could affectsafety for this one or two and
it's important to detect them.
And yeah, I think it depends onthe degree of differences in
(09:22):
this plot if it affects safetyor not, but it can affect.
David Klonoff (09:28):
Are you seeing products coming onto the market
in Europe where there's nopublished data about their
accuracy?
Guido Freckmann (09:36):
Yeah, most of the products coming to the
market have their own data andthe question is how this data
are achieved.
Is there how many rates ofchange, distribution of values
and so on?
So we tested last year a devicethat proposed to have an MRD
(09:57):
below 10 and with a factorycalibrated mode.
It wasn't about double the MRDand with one day calibration it
was still 15.
So I think it's very importantto have comparable procedures to
really see to get comparablevalues, output values out.
David Kerr (10:20):
Yeah, so it seems that's very reassuring for
people with diabetes andclinicians that this accuracy
question needs to be constantlythought about.
I'm sure many of our listenerswould like to know where do you
see non-invasive devices at themoment?
Where are they at the moment interms of accuracy?
(10:40):
Is this really going to happensoon or is this still a pie in
the sky?
Guido Freckmann (10:50):
So my insight into non-invasive devices is
limited, but I know some deviceswhich can already measure I'm
not sure in all devices aroundthere and, interestingly, we
bought two watches who said theycan measure sugar from Asia,
(11:14):
and my colleagues wear thesewatches two days and they show
on each day the same curve.
And on the third day we gave itto a banana and the banana
showed the same curve, like mycolleague.
So these watches are alreadyavailable.
We got them from 50 to 200euros and I think in future we
(11:42):
will have working non-invasivedevices.
But I think, as I know, thecurrent non-invasive devices are
there where CGM was about 10years ago or a little bit more,
and they are developing and theywill come, but I think we need
some more time.
David Klonoff (12:01):
Peter, what advice would you give to a
scientist who wants to develop anew glucose monitor?
Guido Freckmann (12:12):
Yeah, I think many people, especially in the
development of non-invasivedevices, are very optimistic and
they should be, in some case,realistic and looking.
What are the first steps?
You need to go step by step andyou will not have with the
first device, and that's thesame with CGM sensors.
(12:33):
You will need some years toimprove your algorithms and your
device to be in the statuswhere the current CGMs or VCGMs
are.
David Klonoff (12:46):
Is there anything else that you'd like to tell
the audience about yourexperiences or your
recommendations?
Guido Freckmann (12:55):
Yeah, currently .
I already told that wesubmitted a procedure.
We think is very important touse a comparable procedure for
the studies and we are currentlypreparing a study for this and
currently looking for funding,which is not very easy, and I
(13:15):
hope we can get the funding thatwe can perform a study with the
currently well-known devices toset something like a benchmark
with the study.
Where is it if we use moreparameters like CGDVA and not
only MRD, with a really definedstudy procedure?
David Kerr (13:36):
Just a philosophical question.
Is it ever going to be the casethat the difference is going to
be zero, or is it a?
What's the holy grail ofaccuracy for glucose monitoring?
Do we know what that is, or isit just pull it?
Guido Freckmann (13:53):
there.
That's a very good question.
If you measure capillary orvenous you have a difference and
we have seen in health C's inan OGTT post-prandial up to 30%
difference between these twocompartments.
So it's a bit difficult in thebody.
It depends on the place youmeasure.
(14:15):
You will have differences andtherefore the accuracy need to
be for a degree that is, I wouldsay, within 10 to 20% to make
no larger failures.
But it's very difficult to.
You can measure it in a labdevice to a high degree of
accuracy, but on the way to themeasurement are already
(14:38):
differences which you cannot getrid of.
David Klonoff (14:43):
Guido, thank you for speaking with us today.
You're definitely a worldleader in glucose monitoring and
I hope people listen to whatyou had to say so to the
audience.
Thank you for listening to thediabetes technology report.
We're available on Spotify andthe Apple store.
On behalf of David Kerr andmyself, Guido Fragman, thank you
(15:06):
and we will catch up with youat the next podcast.
Bye, everybody.
Thank you very much.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Special Summer Offer: Exclusively on Apple Podcasts, try our Dateline Premium subscription completely free for one month! With Dateline Premium, you get every episode ad-free plus exclusive bonus content.
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy, Jess Hilarious, And Charlamagne Tha God!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.