In this episode, Tom Varghese, MD, FACS, is joined by Brett Johnson, MD, and Clifford Ko, MD, FACS, from the American College of Surgeons (ACS). They discuss the recent article by Drs Johnson and Ko, “Evaluating Outcomes of Initial Site Visits Across American College of Surgeons Accreditation Programs,” in which the authors found that ACS accreditation identifies significant gaps in hospital quality, with only 61% of hospitals passing on their initial attempt. However, most ultimately succeed after remediation. These findings highlight that ACS accreditation both validates hospitals meeting rigorous standards and drives quality improvement in those that initially fall short.
Disclosure Information: Drs Varghese, speaker, has no relevant financial conflicts to disclose. Drs Johnson and Ko, speakers, are employees of the American College of Surgeons.
Learn more about the Journal of the American College of Surgeons, a monthly peer-reviewed journal publishing original contributions on all aspects of surgery, including scientific articles, collective reviews, experimental investigations, and more.
#JACSOperativeWord
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:03):
You are listening to The Operative Word,
a podcast brought to you by the Journalof the American College of Surgeons.
I'm Dr Tom Varghese,and throughout this series, Dr Lillian
Erdahl and I will speak with recentlypublished authors about the motivation
behind their latest researchand the clinical implications
it has for the practicing surgeon.
The opinions expressed in this podcastare those of the participants,
(00:26):
and not necessarilythat of the American College of Surgeons.
Welcome
loyal listenersto another episode of The Operative
Word, the podcast for the Journalof the American College of Surgeons.
My name is Tom Varghese.
I'm the host of today's episode,as well as the editor in chief
for the Journal of the American Collegeof Surgeons.
It gives me great honorto be joined by two amazing
(00:48):
thought leaders in the world of surgery,Dr Clifford Ko and Dr Brett Johnson.
I'm going to have themintroduce yourselves to the audience.
Dr Ko, let's start with you.
Sure. Thanks.
Thanks, Tom. Please call me Cliff.
I'm Clifford Ko.
I'm the director of the Divisionof Research
and Optimal Patient Careat the American College of Surgeons
and professor of surgery at UCLA.
(01:11):
Glad to be here.
And, any relevant disclosuresfor today's episode?
I am the director of the
Division of Research and Optimal PatientCare at the American College of Surgeons.
That's the only relevant disclosures.So nothing financial.
Okay, there you go.
I have just got
Dr Johnson.
Go ahead and introduce yourself.
Absolutely.
(01:31):
First off, thank you, Dr Varghese.
This is really excitingand an honor for me to be here,
to and to discussion,with with two giants in the field.
It's a little intimidating as well.
But this is really exciting for me.
I'm Brett, Johnson,and I'm a current ACS clinical scholar,
and I'm a general surgery resident
at Baylor University,University Medical Center down in Dallas.
(01:52):
I finished two years there.
I'll be going back and starting upmy third year, soon in July.
Thanks so much.
And so, for Cliff and Brett,we're thank you for joining us today.
For the audience, we are discussing,an amazing article that will be
publishing soon, online in the Journalof the American College of Surgeons.
It'll be out online on June 11th
(02:13):
and will be publishedformally in the December issue of JACS.
And the title of the articleis”Evaluating Initial Site Visit Pass Rate
across American College of SurgeonsAccreditation Programs.” Dr Brett
Johnson is the first author of this study,and Dr Clifford Ko
is the senior author for this study.
Let's start with the 30,000-ft view.
(02:34):
Cliff, let's start with you. Why?
What is the importance, do you think ofaccreditation programs in general?
Sure. Great question.
So the accreditation programsof the College of Surgeons is really based
on standards, and this has beenfor decades, the way the College has done
it, standards that are evidencebased, standards that are expert derived,
(02:56):
and standards that we should all adhereto for high quality.
And the College collectsthese standards in different
clinical areastrauma, cancer, bariatric, children's,
and and puts them out there and hopsitalsshould attain these standards.
If they want the highest quality care.
We come in with surveyorswho are trained to evaluate the standards.
(03:18):
And then and then hospitals either passor don't pass.
In terms of adheringand complying to the standards.
Thanks so much, Cliff.
And sometimes I get geeked out aboutthis, for for the audience.
You know, the
College was the first organizationto embrace this idea.
If you really trace back in history,you know, quality standards for hospitals
(03:41):
and other medical facilitieswere first introduced in the United States
by the American Collegeof Surgeons in 1917.
And then formal organizations,accreditation organizations,
such as the JointCommission, came about in 1951
and then subsequently startedspreading across the world.
In the 1980s and 1990s.
And so,I really, is the pride point for all of us
(04:04):
that the College was thereright from the inception.
Let's turn things over to you, Dr Johnson,as you started
digging into this,
article in this this study.
What was the methodology, that you endedup embracing for the current study?
So, the methodology that we embracedwas identifying,
(04:25):
all the initial visits that occurredacross seven ACS accreditation programs,
starting in 2017, through 2023.
And so we identified, over 800 of these,initial visits.
And we looked at whether or notthese initial visits resulted in
full accreditation,meaning that all standards
for the specific program that a hospitalwas pursuing accreditation for were met,
(04:50):
or whether or not, one or more
standards were not metand accreditation was not achieved.
And so and the seven quality programsfor clarification
included the Commission on Cancer,the National Accreditation Program
for Breast Centers, or NAPBC,the National Accreditation Program
for Rectal Cancer, the BariatricAccreditation Quality Improvement, Trauma
(05:13):
Verification, and and the children's,as well as geriatric verification.
Is that correct? Brett?
Yes, sir. That's correct.
Okay. Keep going. Sorry,I didn't mean to interrupt. Go ahead.
And so,
after we identified these initial visitsand determined whether or not
they resulted in accreditation or,not accreditation,
we then followed the all the sitesthat did not achieve accreditation
(05:36):
for their follow up visit to determinewhether or not they achieved accreditation
after going through the remediationprocess where the ACS
identifies and goes identifiesgaps and goes through, goes
through them with the hospitaland gives them,
opportunities to, fix any deficiencies
that were metor that were, found on the initial visit.
(05:59):
And so once we identified, the pass ratesfor initial visits, we then determined
pass rates for reevaluation for the sitesthat did not, meet accreditation.
Out of curiosity.
You know, as Cliff correctly pointed out,these standards are publicly available.
I mean, they're out there.
I mean, they're not hiddenbehind a firewall or anything like that.
And what's the hypothesis going inthat the initial verification rates
(06:23):
would be high,or did you have any a priori
you know, assumptionsgoing into the study?
Yeah, absolutely.
So, whenever we first started going downthe path of this study, we really wanted
to try to get a better understandingof the accreditation process.
We identified a major gap in our knowledgeof how the process works.
Because at the time,
(06:43):
there was no data on initial passrates, across any ACS programs.
And so we asked the question,how many hospitals pass accreditation
on their first attempt?
And our hypothesis was since,as you just pointed out,
all of these standards are publiclyavailable and the criteria are there.
On top of that, the sites have, had
these hospitals have the opportunityto schedule their own site visits.
(07:07):
And so the hypothesis was thatif a site is scheduling
one of these visits,then they believe that they're ready,
to meet all the standards.
And therefore, we expected the,pass rates to be near perfect.
Amazing.
And so let's jump right into the results.
So obviously, as you said, the primaryoutcome was initial accreditation
(07:27):
pass rate.
And the secondary outcome was reevaluationaccreditation pass rate,
meaning those sites that didn't passthe first time, what did they do?
You know, subsequently, going forward.
And so the eye opening statjust leapt right off
the page for me was overall 61%.
So 509 out of 833 hospitals passedtheir initial accreditation site visit.
(07:51):
So that's kind of up to me at least, wasI thought that was
much lower than I thought.
I was expecting, as you said, 80, 90%.
And now we're talking about 60%.
Tell me, Brett,about what happened afterwards.
You know, of those 60%,what about the rest of the 40%?
What what happened withthose hospitals subsequently?
(08:11):
Absolutely.
So, of those 40%, the majority of them,a little bit
more than 80% of them went on to continueto pursue accreditation, meaning
that they went through the remediationprocess.
They, attemptedto try to fix all of their deficiencies,
and they underwent a reevaluation.
12% of the hospitals, 40,
(08:32):
40, 40 hospitalsdecided to withdraw from the program.
And so including those that withdrew,
about 80,a little more than 80% of hospitals
that did not pass their initial visitwent on to, pass their reevaluation.
If you exclude those hospitalsthat immediately withdrew after,
(08:52):
failing to achieve accreditation,initially,
over 90% of those hospitalswent on to achieve accreditation
on their reevaluation visit,which really emphasizes,
the point that, the the process works and,
ACS accreditationwhile it, it's a marker of excellence.
It's also a, has a dual capacityof being a catalyst to really improve
(09:15):
qualityand, quality across these hospitals and,
these hospitalswho failed initially achieving a success
rate of over 90%, really speaksto how well, these hospitals engage in how
well the ACS engages with the hospitalsthat need a little bit of extra help.
I guess, this question, I'll lob at Cliff,because we got to make sure
(09:36):
that we give himthe tougher questions, Brett.
So I appreciate that.
Cliff, was that a surprise to you as well?
60% or what do you think about this?
I mean, it's like.
Yeah,you know, is that what's to be expected?
Like, if we were to do these typeof studies regularly every few years,
do you expect that to always be aroundthe 60% in terms of initial pass rate.
(10:00):
Yeah, I think so.
That is a great question, Tom.
And and you know, I thinkwhat we're trying to do is to address
this notion in, across the landscape that,
everyone's good.
And, and there is a fallacy that, that,
you know, you just sign upwith the College pay your whatever
(10:21):
small fee, and then you passand then you get,
you get accredited and,
I'm not I don't think that that's the waythe College quality programs work.
Maybe other accreditation programswork like that, but not the College.
And so I thought thatthis was a really interesting study to me
because if we take the 5000hospitals in the country,
(10:43):
how many of them de novowould be accredited?
And this tells us that evenwhen the standards are public, just like
you said, people who go on to the websiteand look at the standards
and when a surveyor comesin, a third party comes in
to say, you know, you meetor you don't meet these standards.
What is the pass rate?
And even when they have the test,they have the test questions.
(11:06):
They prepare for the teststill 40%, almost 40% fail.
And so can you imagine, Tom,what it is out
in the rest of the landscapewhen they don't prepare for the test?
And what type of care that they're giving?
It's probably much,
much higher than a 40% fail rate.
So at the College,we really are trying to get every patient,
(11:29):
every surgical patientto have high-quality care.
Studies have shownthat the accreditation process
that the College does leaddoes lead to better quality.
And and this showsthat not everyone would pass.
If they're just kind of doingwhat they think is best, going,
undergoing the verificationor accreditation process really helps.
(11:50):
It's a, it's incredible.
I mean, I'm going to throw a couple of,historical relevance here.
Ernest Codman,of course, is a legendary figure
in the historyof the American College of Surgeons,
because he was one of the first pioneersout there saying that every, every surgeon
and every hospital needs to be meticulousabout recording their outcomes.
(12:12):
Now, Dr Codman ran into problemsbecause he got pissed off at everybody
and subsequently drew a cartoonof an ostrich, with his head in the sand,
saying that this is the waymost hospitals and health systems are now.
Again, I'm dating myself.
This is back in the 19early 19 aughts and 19 tens, 1920s.
(12:33):
I guess, Cliff, the question is, is that,you know, we're all talking to each other
and we're preaching to the choir, butwhat would your message be to the C-suite?
You know, and say that,okay, this is a study that we have.
What what what message would you say
that everybody needs to understandwhat the reality is?
I think that rightnow, the C-suite and front-line
(12:53):
surgeons are facing a new reality of,
having to produce high-qualitycare, efficiently, effectively.
And, and really,while looking at the bottom line of cost
and by decreasing waste,
by increasing standardization,we'll get the efficiency.
(13:14):
Having it donewell, we'll get to the effectiveness.
And that will lead to better,higher value.
And these programs help hospitals do that.
So whether you're C-suiteand having to kind of do this and you're
always looking at the spreadsheetsand the bottom line, this will help.
If you are a front-line surgeonand you want your hospital to be well
resourced in order for you to providehigh quality care for your patients,
(13:38):
these programs will help.
So I think it's kindof one of those win-win-wins.
But it does take work,
as we know that even these hospitalsthat prepare and kind of get ready,
if 40% don't pass on that first go round,it does take work.
But what doesn't take workif you want to have something good.
But what doesn’t I mean?
There's no free lunch out there.
(13:58):
There's no.
That's amazing.
Well, I'm going to Brett, I'm going to askyou a question, and then, end,
wrap things up. We're going back to Cliff.
But, Brett,
I'm going to read you
a couple sentences directlyfrom the conclusions of your paper.
So, “This study highlightsthe dual role of ACS accreditation
in both validating hospitalsthat meet rigorous quality standards
(14:21):
and serving as a driverfor systemic quality improvement.
Among those that initially fall short.”Obviously, that's
reflective of the fact that,as you said, 80 to 90% of those hospitals
that didn't meet the initial accreditationstandard did so subsequently.
They put the work inand then got got the accreditation.
The question, though, is like the,
(14:43):
as Cliff alluded to, and this is for you,Brett, “Given the likelihood
of even greater deficiencies in hospitalsnot actively seeking accreditation,”
you know, the unknown out there, “futureresearch
should explore strategies to expandaccreditation efforts and provide
additional quality improvement resourcesto reduce disparities in surgical care.”
(15:05):
As a surgical trainee, Brett,
how did this study influenceyour thought process
or what are your plans for the futurein terms of the types of studies
that you wish to embark on
as directly as a resultof what you've learned from this study?
Thanks for that question.
It's an exciting question for me,because I feel like there's
a lot of avenuesthat that this study opens up
and doors that this opens upto start to pursue further.
(15:29):
And studies that really inform,
how the ACS accreditation process works.
One particular, study,
which you, alluded to is understanding
how, non, why non participation happens.
So there's plenty of studies out there,growing number of studies
(15:49):
that show that ACS accreditationis associated with better outcomes.
Better clinical care.
Even better efficiency.
Our study herealso demonstrates that not only does,
ACS accreditation identify
hospitals, achieving high quality care,it also identifies gaps in hospitals
and helps them to achieve high qualitycare after,
(16:13):
improving on deficienciesthat were identified.
So with all this evidence,why are hospitals
not taking advantage of an opportunityto improve care within their hospital?
I think there's, certainly barriersthat, many
maybe identified and others that areunidentified that are preventing hospitals
from engaging in that, process,because why else would they not be?
(16:36):
There's there's a strong correlationthat if they engage in the process.
The ACS accreditation
process will ultimately lead to betteroutcomes, better efficiency.
And so trying to identify those barriers,trying to break down those, walls
and trying to get more hospitalsto engage in the process
so that quality can continueto improve at a systemic level.
(17:00):
That's well said.
Brett, thank you for saying that.
Now I'm going to pivot back to Cliff.
Like I said, I always leave the harderquestions for Cliff.
I'm just kidding, Cliff.
So, so obviously the holy grailfor all of us.
Ultimately, for many people in the qualityworld is public reporting.
(17:21):
And traditionally accreditationhas been aimed at organizations.
But the question then comes is, what aboutthe individual practicing surgeons?
Because we know thatthere are some surgeons
that are tied to a hospital systemand they're one place,
and then there are other surgeonsthat we know that are
taking care of patientsin multiple different facilities.
What what what do you think?
(17:43):
We should go towards, Cliff,in terms of using this study
is kind of the initial foray into this,but what do you think the future is?
Yeah.
Tough question. Thank you very much, Tom.
So, you know, at the College,when we do analyses
and when we have data,the first thing, we think about is
(18:04):
we have to make surethat we're doing valid studies.
And when we report something, there hasto be validity, and we do not ever,
want to inappropriately or.
Yeah, or inappropriately ding somebodywhen they're not,
you know, when they don't deserve it.
But what do we not only look at hospitalsversus individual surgeons,
the individual surgeonsare very difficult to evaluate because of,
(18:28):
the, the the detail of the metrics,
the attribution of the metricand the sample size.
And when we don't have enough sample size,then it's very difficult
to say who's good, who's better,who's worse, and so forth.
At the individual level.
So that's why we have a lot of,
our quality programs are hospital basedand we have the sample size.
(18:50):
We do think that there's validityin that since, most surgeons
at least that that, that are membersof the College and fellows of the College
are employed by hospitals.
And so being hospital based,we believe that the hospitals
responsibility is to make sure that,
you know, not just the surgeons,but anesthesia and the nursing
and the pharmacies or whatever,are working together as a team.
(19:12):
And so the surgeonand the surgical team are very important.
And when we accredit a hospital, we lookat those things, we look at who's on call,
you know, there's athere's a call schedule for,
for these certain things for the surgeon.
In our children's program,
we are very cognizant of the importanceof a pediatric anesthesiologist.
So that piece is there,we're cognizant of the nursing,
(19:35):
and the and other staffthat make up the surgical team.
So that'swhy we, evaluate these hospitals.
At that unit of study.
You mentioned that public reportingand, I'd be remiss
if I didn't mentionthat our integrated communications
folks are developing right now,and it'll be released in July.
(19:56):
A website,a publicly reporting website of hospitals
that have achieved accreditation statusfrom the American College of Surgeons.
So it's going to belike one of those websites,
that if you want to if you come to Chicagoand you want to find a restaurant,
you look at this map and like,oh, there's a restaurant within,
you know, five feet of me or 100ft of meor five miles from me.
(20:17):
And the rating of that of that restaurant,this is very similar, but for hospitals
and who have achieved the accreditationstatus of
going through all of these thingsand passing all the all the standards,
and it's going to be on on that websiteand it'll it'll show that, as you know,
the quality diamonds, the quality partnerdiamonds of the College on this website.
(20:39):
To know that.
Oh, I want to my, you know, we need,
a hospitalthat is expert in children's surgery.
Where are the closest facilitiesthat meet that?
Those standards and have a qualitydiamond within, you know, 20 miles of me?
And you can go to the website and do that.
So that's really patient facing.
But I think also think it'sgoing to be facing
(21:02):
our referrals from primary careand so forth.
That's amazing.
Cliff, any final words in terms of,you know, at the time of this recording,
it's just before the July,
ACS Quality and Safety meeting.
Any other highlightsthat we should be thinking about?
I mean, obviously that website isthat's that's exciting.
But anything else, we should keep it,keep our eyes up for?
(21:25):
maybe a couple teasers.
One, when this comes outin electronic form or the published
form, people should read itbecause, in the study we looked at
and we identified
different types of standards that arethat are the difficult ones that,
that, that might are,it might be more difficult to pass.
So people should look at that
(21:45):
if they're kind of going downthe process of accreditation.
And the second thing is,if hospitals are doing that,
come to the Quality Safety Conferencein July, it's in San Diego.
That's a great placeto network with other hospitals
and other leaders and other people goingthrough the same process to learn lessons.
So you don't have to reinvent the wheelof kind of going through the process
(22:07):
and figuring out,
what, what are key things to know
about to,to improve the chance of passing.
Amazing, amazing work.
Well, I'm honored to have been joinedby both Dr Clifford Ko
and Dr Brett Johnson todayfor today's episode.
As was mentioned, please,keep an eye out for this
(22:27):
article will be publishedonline on June 11th.
On behalf of our listeners,
thank you both for joining us on today'sepisode of The Operative Word,
a podcast for the Journalof the American College of Surgeons.
Thanks so much.
Thank you for listening
to the Journal of the American Collegeof Surgeons Operative Word Podcast.
(22:48):
If you enjoyed today's episode,spread the word on social media
by using the hashtag #JACSOperativeWord.
Subscribe to The Operative Wordwherever podcasts are available,
or listen on the American Collegeof Surgeons website at FACS.org/podcast.
You are listening to The Operative Word,
a podcast brought to you by the Journalof the American College of Surgeons.
I'm Dr Tom Varghese,and throughout this series, Dr Lillian
Erdahl and I will speak with recentlypublished authors about the motivation
behind their latest researchand the clinical implications
it has for the practicing surgeon.
The opinions expressed in this podcastare those of the participants,
(00:26):
and not necessarilythat of the American College of Surgeons.
Welcome
loyal listenersto another episode of The Operative
Word, the podcast for the Journalof the American College of Surgeons.
My name is Tom Varghese.
I'm the host of today's episode,as well as the editor in chief
for the Journal of the American Collegeof Surgeons.
It gives me great honorto be joined by two amazing
(00:48):
thought leaders in the world of surgery,Dr Clifford Ko and Dr Brett Johnson.
I'm going to have themintroduce yourselves to the audience.
Dr Ko, let's start with you.
Sure. Thanks.
Thanks, Tom. Please call me Cliff.
I'm Clifford Ko.
I'm the director of the Divisionof Research
and Optimal Patient Careat the American College of Surgeons
and professor of surgery at UCLA.
(01:11):
Glad to be here.
And, any relevant disclosuresfor today's episode?
I am the director of the
Division of Research and Optimal PatientCare at the American College of Surgeons.
That's the only relevant disclosures.So nothing financial.
Okay, there you go.
I have just got
Dr Johnson.
Go ahead and introduce yourself.
Absolutely.
(01:31):
First off, thank you, Dr Varghese.
This is really excitingand an honor for me to be here,
to and to discussion,with with two giants in the field.
It's a little intimidating as well.
But this is really exciting for me.
I'm Brett, Johnson,and I'm a current ACS clinical scholar,
and I'm a general surgery resident
at Baylor University,University Medical Center down in Dallas.
(01:52):
I finished two years there.
I'll be going back and starting upmy third year, soon in July.
Thanks so much.
And so, for Cliff and Brett,we're thank you for joining us today.
For the audience, we are discussing,an amazing article that will be
publishing soon, online in the Journalof the American College of Surgeons.
It'll be out online on June 11th
(02:13):
and will be publishedformally in the December issue of JACS.
And the title of the articleis”Evaluating Initial Site Visit Pass Rate
across American College of SurgeonsAccreditation Programs.” Dr Brett
Johnson is the first author of this study,and Dr Clifford Ko
is the senior author for this study.
Let's start with the 30,000-ft view.
(02:34):
Cliff, let's start with you. Why?
What is the importance, do you think ofaccreditation programs in general?
Sure. Great question.
So the accreditation programsof the College of Surgeons is really based
on standards, and this has beenfor decades, the way the College has done
it, standards that are evidencebased, standards that are expert derived,
(02:56):
and standards that we should all adhereto for high quality.
And the College collectsthese standards in different
clinical areastrauma, cancer, bariatric, children's,
and and puts them out there and hopsitalsshould attain these standards.
If they want the highest quality care.
We come in with surveyorswho are trained to evaluate the standards.
(03:18):
And then and then hospitals either passor don't pass.
In terms of adheringand complying to the standards.
Thanks so much, Cliff.
And sometimes I get geeked out aboutthis, for for the audience.
You know, the
College was the first organizationto embrace this idea.
If you really trace back in history,you know, quality standards for hospitals
(03:41):
and other medical facilitieswere first introduced in the United States
by the American Collegeof Surgeons in 1917.
And then formal organizations,accreditation organizations,
such as the JointCommission, came about in 1951
and then subsequently startedspreading across the world.
In the 1980s and 1990s.
And so,I really, is the pride point for all of us
(04:04):
that the College was thereright from the inception.
Let's turn things over to you, Dr Johnson,as you started
digging into this,
article in this this study.
What was the methodology, that you endedup embracing for the current study?
So, the methodology that we embracedwas identifying,
(04:25):
all the initial visits that occurredacross seven ACS accreditation programs,
starting in 2017, through 2023.
And so we identified, over 800 of these,initial visits.
And we looked at whether or notthese initial visits resulted in
full accreditation,meaning that all standards
for the specific program that a hospitalwas pursuing accreditation for were met,
(04:50):
or whether or not, one or more
standards were not metand accreditation was not achieved.
And so and the seven quality programsfor clarification
included the Commission on Cancer,the National Accreditation Program
for Breast Centers, or NAPBC,the National Accreditation Program
for Rectal Cancer, the BariatricAccreditation Quality Improvement, Trauma
(05:13):
Verification, and and the children's,as well as geriatric verification.
Is that correct? Brett?
Yes, sir. That's correct.
Okay. Keep going. Sorry,I didn't mean to interrupt. Go ahead.
And so,
after we identified these initial visitsand determined whether or not
they resulted in accreditation or,not accreditation,
we then followed the all the sitesthat did not achieve accreditation
(05:36):
for their follow up visit to determinewhether or not they achieved accreditation
after going through the remediationprocess where the ACS
identifies and goes identifiesgaps and goes through, goes
through them with the hospitaland gives them,
opportunities to, fix any deficiencies
that were metor that were, found on the initial visit.
(05:59):
And so once we identified, the pass ratesfor initial visits, we then determined
pass rates for reevaluation for the sitesthat did not, meet accreditation.
Out of curiosity.
You know, as Cliff correctly pointed out,these standards are publicly available.
I mean, they're out there.
I mean, they're not hiddenbehind a firewall or anything like that.
And what's the hypothesis going inthat the initial verification rates
(06:23):
would be high,or did you have any a priori
you know, assumptionsgoing into the study?
Yeah, absolutely.
So, whenever we first started going downthe path of this study, we really wanted
to try to get a better understandingof the accreditation process.
We identified a major gap in our knowledgeof how the process works.
Because at the time,
(06:43):
there was no data on initial passrates, across any ACS programs.
And so we asked the question,how many hospitals pass accreditation
on their first attempt?
And our hypothesis was since,as you just pointed out,
all of these standards are publiclyavailable and the criteria are there.
On top of that, the sites have, had
these hospitals have the opportunityto schedule their own site visits.
(07:07):
And so the hypothesis was thatif a site is scheduling
one of these visits,then they believe that they're ready,
to meet all the standards.
And therefore, we expected the,pass rates to be near perfect.
Amazing.
And so let's jump right into the results.
So obviously, as you said, the primaryoutcome was initial accreditation
(07:27):
pass rate.
And the secondary outcome was reevaluationaccreditation pass rate,
meaning those sites that didn't passthe first time, what did they do?
You know, subsequently, going forward.
And so the eye opening statjust leapt right off
the page for me was overall 61%.
So 509 out of 833 hospitals passedtheir initial accreditation site visit.
(07:51):
So that's kind of up to me at least, wasI thought that was
much lower than I thought.
I was expecting, as you said, 80, 90%.
And now we're talking about 60%.
Tell me, Brett,about what happened afterwards.
You know, of those 60%,what about the rest of the 40%?
What what happened withthose hospitals subsequently?
(08:11):
Absolutely.
So, of those 40%, the majority of them,a little bit
more than 80% of them went on to continueto pursue accreditation, meaning
that they went through the remediationprocess.
They, attemptedto try to fix all of their deficiencies,
and they underwent a reevaluation.
12% of the hospitals, 40,
(08:32):
40, 40 hospitalsdecided to withdraw from the program.
And so including those that withdrew,
about 80,a little more than 80% of hospitals
that did not pass their initial visitwent on to, pass their reevaluation.
If you exclude those hospitalsthat immediately withdrew after,
(08:52):
failing to achieve accreditation,initially,
over 90% of those hospitalswent on to achieve accreditation
on their reevaluation visit,which really emphasizes,
the point that, the the process works and,
ACS accreditationwhile it, it's a marker of excellence.
It's also a, has a dual capacityof being a catalyst to really improve
(09:15):
qualityand, quality across these hospitals and,
these hospitalswho failed initially achieving a success
rate of over 90%, really speaksto how well, these hospitals engage in how
well the ACS engages with the hospitalsthat need a little bit of extra help.
I guess, this question, I'll lob at Cliff,because we got to make sure
(09:36):
that we give himthe tougher questions, Brett.
So I appreciate that.
Cliff, was that a surprise to you as well?
60% or what do you think about this?
I mean, it's like.
Yeah,you know, is that what's to be expected?
Like, if we were to do these typeof studies regularly every few years,
do you expect that to always be aroundthe 60% in terms of initial pass rate.
(10:00):
Yeah, I think so.
That is a great question, Tom.
And and you know, I thinkwhat we're trying to do is to address
this notion in, across the landscape that,
everyone's good.
And, and there is a fallacy that, that,
you know, you just sign upwith the College pay your whatever
(10:21):
small fee, and then you passand then you get,
you get accredited and,
I'm not I don't think that that's the waythe College quality programs work.
Maybe other accreditation programswork like that, but not the College.
And so I thought thatthis was a really interesting study to me
because if we take the 5000hospitals in the country,
(10:43):
how many of them de novowould be accredited?
And this tells us that evenwhen the standards are public, just like
you said, people who go on to the websiteand look at the standards
and when a surveyor comesin, a third party comes in
to say, you know, you meetor you don't meet these standards.
What is the pass rate?
And even when they have the test,they have the test questions.
(11:06):
They prepare for the teststill 40%, almost 40% fail.
And so can you imagine, Tom,what it is out
in the rest of the landscapewhen they don't prepare for the test?
And what type of care that they're giving?
It's probably much,
much higher than a 40% fail rate.
So at the College,we really are trying to get every patient,
(11:29):
every surgical patientto have high-quality care.
Studies have shownthat the accreditation process
that the College does leaddoes lead to better quality.
And and this showsthat not everyone would pass.
If they're just kind of doingwhat they think is best, going,
undergoing the verificationor accreditation process really helps.
(11:50):
It's a, it's incredible.
I mean, I'm going to throw a couple of,historical relevance here.
Ernest Codman,of course, is a legendary figure
in the historyof the American College of Surgeons,
because he was one of the first pioneersout there saying that every, every surgeon
and every hospital needs to be meticulousabout recording their outcomes.
(12:12):
Now, Dr Codman ran into problemsbecause he got pissed off at everybody
and subsequently drew a cartoonof an ostrich, with his head in the sand,
saying that this is the waymost hospitals and health systems are now.
Again, I'm dating myself.
This is back in the 19early 19 aughts and 19 tens, 1920s.
(12:33):
I guess, Cliff, the question is, is that,you know, we're all talking to each other
and we're preaching to the choir, butwhat would your message be to the C-suite?
You know, and say that,okay, this is a study that we have.
What what what message would you say
that everybody needs to understandwhat the reality is?
I think that rightnow, the C-suite and front-line
(12:53):
surgeons are facing a new reality of,
having to produce high-qualitycare, efficiently, effectively.
And, and really,while looking at the bottom line of cost
and by decreasing waste,
by increasing standardization,we'll get the efficiency.
(13:14):
Having it donewell, we'll get to the effectiveness.
And that will lead to better,higher value.
And these programs help hospitals do that.
So whether you're C-suiteand having to kind of do this and you're
always looking at the spreadsheetsand the bottom line, this will help.
If you are a front-line surgeonand you want your hospital to be well
resourced in order for you to providehigh quality care for your patients,
(13:38):
these programs will help.
So I think it's kindof one of those win-win-wins.
But it does take work,
as we know that even these hospitalsthat prepare and kind of get ready,
if 40% don't pass on that first go round,it does take work.
But what doesn't take workif you want to have something good.
But what doesn’t I mean?
There's no free lunch out there.
(13:58):
There's no.
That's amazing.
Well, I'm going to Brett, I'm going to askyou a question, and then, end,
wrap things up. We're going back to Cliff.
But, Brett,
I'm going to read you
a couple sentences directlyfrom the conclusions of your paper.
So, “This study highlightsthe dual role of ACS accreditation
in both validating hospitalsthat meet rigorous quality standards
(14:21):
and serving as a driverfor systemic quality improvement.
Among those that initially fall short.”Obviously, that's
reflective of the fact that,as you said, 80 to 90% of those hospitals
that didn't meet the initial accreditationstandard did so subsequently.
They put the work inand then got got the accreditation.
The question, though, is like the,
(14:43):
as Cliff alluded to, and this is for you,Brett, “Given the likelihood
of even greater deficiencies in hospitalsnot actively seeking accreditation,”
you know, the unknown out there, “futureresearch
should explore strategies to expandaccreditation efforts and provide
additional quality improvement resourcesto reduce disparities in surgical care.”
(15:05):
As a surgical trainee, Brett,
how did this study influenceyour thought process
or what are your plans for the futurein terms of the types of studies
that you wish to embark on
as directly as a resultof what you've learned from this study?
Thanks for that question.
It's an exciting question for me,because I feel like there's
a lot of avenuesthat that this study opens up
and doors that this opens upto start to pursue further.
(15:29):
And studies that really inform,
how the ACS accreditation process works.
One particular, study,
which you, alluded to is understanding
how, non, why non participation happens.
So there's plenty of studies out there,growing number of studies
(15:49):
that show that ACS accreditationis associated with better outcomes.
Better clinical care.
Even better efficiency.
Our study herealso demonstrates that not only does,
ACS accreditation identify
hospitals, achieving high quality care,it also identifies gaps in hospitals
and helps them to achieve high qualitycare after,
(16:13):
improving on deficienciesthat were identified.
So with all this evidence,why are hospitals
not taking advantage of an opportunityto improve care within their hospital?
I think there's, certainly barriersthat, many
maybe identified and others that areunidentified that are preventing hospitals
from engaging in that, process,because why else would they not be?
(16:36):
There's there's a strong correlationthat if they engage in the process.
The ACS accreditation
process will ultimately lead to betteroutcomes, better efficiency.
And so trying to identify those barriers,trying to break down those, walls
and trying to get more hospitalsto engage in the process
so that quality can continueto improve at a systemic level.
(17:00):
That's well said.
Brett, thank you for saying that.
Now I'm going to pivot back to Cliff.
Like I said, I always leave the harderquestions for Cliff.
I'm just kidding, Cliff.
So, so obviously the holy grailfor all of us.
Ultimately, for many people in the qualityworld is public reporting.
(17:21):
And traditionally accreditationhas been aimed at organizations.
But the question then comes is, what aboutthe individual practicing surgeons?
Because we know thatthere are some surgeons
that are tied to a hospital systemand they're one place,
and then there are other surgeonsthat we know that are
taking care of patientsin multiple different facilities.
What what what do you think?
(17:43):
We should go towards, Cliff,in terms of using this study
is kind of the initial foray into this,but what do you think the future is?
Yeah.
Tough question. Thank you very much, Tom.
So, you know, at the College,when we do analyses
and when we have data,the first thing, we think about is
(18:04):
we have to make surethat we're doing valid studies.
And when we report something, there hasto be validity, and we do not ever,
want to inappropriately or.
Yeah, or inappropriately ding somebodywhen they're not,
you know, when they don't deserve it.
But what do we not only look at hospitalsversus individual surgeons,
the individual surgeonsare very difficult to evaluate because of,
(18:28):
the, the the detail of the metrics,
the attribution of the metricand the sample size.
And when we don't have enough sample size,then it's very difficult
to say who's good, who's better,who's worse, and so forth.
At the individual level.
So that's why we have a lot of,
our quality programs are hospital basedand we have the sample size.
(18:50):
We do think that there's validityin that since, most surgeons
at least that that, that are membersof the College and fellows of the College
are employed by hospitals.
And so being hospital based,we believe that the hospitals
responsibility is to make sure that,
you know, not just the surgeons,but anesthesia and the nursing
and the pharmacies or whatever,are working together as a team.
(19:12):
And so the surgeonand the surgical team are very important.
And when we accredit a hospital, we lookat those things, we look at who's on call,
you know, there's athere's a call schedule for,
for these certain things for the surgeon.
In our children's program,
we are very cognizant of the importanceof a pediatric anesthesiologist.
So that piece is there,we're cognizant of the nursing,
(19:35):
and the and other staffthat make up the surgical team.
So that'swhy we, evaluate these hospitals.
At that unit of study.
You mentioned that public reportingand, I'd be remiss
if I didn't mentionthat our integrated communications
folks are developing right now,and it'll be released in July.
(19:56):
A website,a publicly reporting website of hospitals
that have achieved accreditation statusfrom the American College of Surgeons.
So it's going to belike one of those websites,
that if you want to if you come to Chicagoand you want to find a restaurant,
you look at this map and like,oh, there's a restaurant within,
you know, five feet of me or 100ft of meor five miles from me.
(20:17):
And the rating of that of that restaurant,this is very similar, but for hospitals
and who have achieved the accreditationstatus of
going through all of these thingsand passing all the all the standards,
and it's going to be on on that websiteand it'll it'll show that, as you know,
the quality diamonds, the quality partnerdiamonds of the College on this website.
(20:39):
To know that.
Oh, I want to my, you know, we need,
a hospitalthat is expert in children's surgery.
Where are the closest facilitiesthat meet that?
Those standards and have a qualitydiamond within, you know, 20 miles of me?
And you can go to the website and do that.
So that's really patient facing.
But I think also think it'sgoing to be facing
(21:02):
our referrals from primary careand so forth.
That's amazing.
Cliff, any final words in terms of,you know, at the time of this recording,
it's just before the July,
ACS Quality and Safety meeting.
Any other highlightsthat we should be thinking about?
I mean, obviously that website isthat's that's exciting.
But anything else, we should keep it,keep our eyes up for?
(21:25):
maybe a couple teasers.
One, when this comes outin electronic form or the published
form, people should read itbecause, in the study we looked at
and we identified
different types of standards that arethat are the difficult ones that,
that, that might are,it might be more difficult to pass.
So people should look at that
(21:45):
if they're kind of going downthe process of accreditation.
And the second thing is,if hospitals are doing that,
come to the Quality Safety Conferencein July, it's in San Diego.
That's a great placeto network with other hospitals
and other leaders and other people goingthrough the same process to learn lessons.
So you don't have to reinvent the wheelof kind of going through the process
(22:07):
and figuring out,
what, what are key things to know
about to,to improve the chance of passing.
Amazing, amazing work.
Well, I'm honored to have been joinedby both Dr Clifford Ko
and Dr Brett Johnson todayfor today's episode.
As was mentioned, please,keep an eye out for this
(22:27):
article will be publishedonline on June 11th.
On behalf of our listeners,
thank you both for joining us on today'sepisode of The Operative Word,
a podcast for the Journalof the American College of Surgeons.
Thanks so much.
Thank you for listening
to the Journal of the American Collegeof Surgeons Operative Word Podcast.
(22:48):
If you enjoyed today's episode,spread the word on social media
by using the hashtag #JACSOperativeWord.
Subscribe to The Operative Wordwherever podcasts are available,
or listen on the American Collegeof Surgeons website at FACS.org/podcast.
Popular Podcasts
United States of Kennedy
United States of Kennedy is a podcast about our cultural fascination with the Kennedy dynasty. Every week, hosts Lyra Smith and George Civeris go into one aspect of the Kennedy story.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com