October 31, 2023 • 49 mins
This episode was recorded live from the KENX Medical Device Validation University in Anaheim, CA! This time around, Jason Secola interviews your usual host, Dori Gonzalez-Acevedo to give a conference recap, as well as field some questions submitted by our LinkedIn audience.
Tune in as they dive into a variety of topics, not least of which is the world of AI and Software Quality in Medical Devices and the Life Sciences industry. Some key points discussed were the FDA's increasing approval of AI-powered devices, the importance of collaborative education, the criticality of developing a Quality by Design program, and the impact of diverse perspectives in the Life Sciences industry.
Discover the keys to innovation, efficient processes, and nuanced risk management in this insightful conversation with Dori Gonzalez-Acevedo and Jason Secola.
Share with a friend or colleague and subscribe wherever listen to podcasts!
*Disclaimer: Podcast guest participated in the podcast as an individual subject matter expert and contributor. The views and opinions they share are not necessarily shared by their employer. Nor should any reference to specific products or services be interpreted as commercial endorsements by their current employer.
This is a production of ProcellaRX
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Jason Secola (00:00):
Okay, so welcome to another episode of software
quality today. I'm Jason Secola.
I'm going to be taking overhosting duties for this episode
anyways, and I'm with
Dori Gonzalez-Acevedo (00:10):
Dori Gonzalez Acevedo your usual
host.
Jason Secola (00:12):
Yes. So what we're going to do today is we're
actually out in the Kanex,medical device validation
University in Anaheim, rightoutside of the gates of the
Magic Kingdom.
Dori Gonzalez-Acevedo (00:25):
Yes, we are. We're just literally a walk
away. Yeah.
Jason Secola (00:28):
So we thought we'd do a little bit of a conference
recap, while we're here. And wealso a couple of weeks ago, put
out for a request for somequestions on LinkedIn for some
folks that we thought maybe youcould spend some time going
over. So we've got all thatstuff queued up. But first of
all, since we're here, sincewe're through two days now of
(00:50):
the conference, I thought maybewe'd kind of dive into what
people are talking about whatwe're hearing about, I know you
let a sort of an impromptu panelthis morning to help adjust for
some folks not showing up thatwe're supposed to be presenting.
You did a solo presentation, butwe also had a chance to sit in
on on a bunch of panels andother presentations as well. So
(01:11):
as there's been pretty common, Ithink at a lot of recent shows
and webinars, AI seems to be avery prevalent theme. A lot of
typical stuff that we see aroundmed device, software's medical
device, validation applications,software, quality applications,
and then broader applications,right. But what do you want to
start with? What were your somesome of your key takeaways?
Dori Gonzalez-Acevedo (01:32):
Well, first, this was a, the first
medical device university thatcan access done solely targeted
at the medical device community.
So for context for everyone,this was, I'm not sure numbers
wise, but it was a small,intimate sort of grouping that's
and folks that actually areproducing medical devices, or
(01:54):
software as a medical device,which is a different audience
than than we've had in the past.
Connect. So I think that that'sreally important. When we kind
of did a show of hands of howmany quality people were in the
room, hands went up galore. Andat the same time, when I asked,
you know, how many engineeringfolks are in the room, a lot of
(02:16):
folks also raise their hands.
Right. So I think it was aunique mix of folks, because a
lot of them play multiple rolesor having to do multiple roles
within their organizations,because they're small, some
small to midsize companies. Butat the same time, we're wearing
lots of hats
Jason Secola (02:37):
seems to be more and more commonplace, regardless
of size of company. Yeah, itseems like the expectation at a
lot of these organizations ismulti skill, multifaceted
hybrid, and it's kind of thenature of how it kind of has to
be in a way. Right.
Dori Gonzalez-Acevedo (02:52):
Yeah. So I think that's important just
for everyone to kind of know.
And then we had, you know, ourpanels in our speakers are from
a variety of different companiesthat are actually doing medical
device manufacturing. And DanielWalter from the FDA was with us,
which was great to be able tokind of give that perspective,
(03:13):
from a regulatory bodyexpectation, which has been
fabulous.
Jason Secola (03:20):
It is it's been really good to see. You know,
obviously, Cisco has been at alot of stuff, Daniel, I've seen
it a lot of stuff, and theirwillingness and want to get out
there and sort of let thecommunity know that they're open
to discussion. They're open toquestions, they're open, be
brought in proactively, theywant to drive innovation, and
(03:42):
not be seen as stiflinginnovation. And their regular
presence at these things, Ithink, kind of illustrates that.
Whether or not you know, peopleare embracing them on that. I
think that varies, but yeah,
Dori Gonzalez-Aceve (03:53):
consistent.
All right, their message isreach out proactively. And in
the Met. Again, this is slightlydifferent than the typical, you
know, connects when we have ageneral validation University
conversation where we're havinga broader mix of drug product,
as well as medical device. Andalso, in my experience, a lot of
medical device companies tend tobe a little bit more forward
(04:16):
thinking in their softwarequality, about how to do that,
because they've had to do thatfor medical device that they
have. And so whether or not it'sa standalone device that has no
software embedded or is that hassome software, but the way they
think about this is more of asoftware engineering mindset.
(04:37):
Then some of the other broaderorganizations that don't
necessarily have that as a corecompetency yet, I say yet
because a lot of what we'reseeing right is more compound or
complex products that are beingbrought up to market right and
that Then the necessity to beable to do this more
(05:02):
collaboratively is in everyone'sbest interest. But you're right.
AI machine learning is a hottopic today. And what I
illustrated in the panel todaywas, again, the first algorithm
was approved in 1995. Yeah.
(05:24):
Right. And there was a very,very slow progression in
algorithms since that time. Butthis has been part of what has
been happening for many years.
And so now, as more broad,broader and more mainstream, the
topic is, right, we're seeingthat more and more people have
(05:46):
to be educated on it andunderstand the role that it
plays, whether it's a part of aproduct or not, you know, what
is the intended use of usingsome of these for my use case
perspective? And being reallyclear about that? And I think
that that we saw that strugglebefore. And I think we're going
(06:07):
to have to double down on how dowe make that better this time
around?
Jason Secola (06:14):
Yeah, I mean, it's a little bit clearer in the
questions people are askingsometimes what they're posing.
And it was also, I don't know ifit was surprising, because it
was a little bit surprising tome when you brought up some of
the stats last night when wewere at dinner about all the
algorithms and other things thathave already been approved by
the FDA. And the number was alot higher than I would have
(06:34):
thought. But that was clear inthe room. Also, when you asked
that I think people thoughtnothing was approved this year,
up to January 2023, or whateverthat date was zero.
Dori Gonzalez-Acevedo (06:44):
And it's not even close. Yeah. Right. And
so this is partly education. Andthe FDA is, you know, been so
transparent through this wholeprocess. It's just whether or
not folks had knew that thatinformation was available,
right? And maybe now becausewe're all talking about it,
we're going to take the nextstep to educate within our own
(07:05):
organizations, and then how todo this and how to apply it.
Where does it apply? Where doesit not apply? And to not my
hope, is that we don't overengineer yet again, and go down
a path that brought us to thisoverburden validation process of
CSV that we have today, in manyorganizations, and we learn a
(07:27):
lesson. And we actually disruptat organization levels to do
something different.
Jason Secola (07:34):
Yeah, well, that kind of goes in line with what
in one of the panels thathappened today, there were some
there's a little bit of back andforth between the consultant
side and the customer side andknowledge gaps and needing to
make decisions. So I think, toyour point, a lot of that stuff
has to do with people need to bemore comfortable. I think
(07:56):
leveraging people on theconsultancy side who have
dedicated expertise in thisstuff, or who are more forward
thinking, trust their judgment,don't get that like heels dug in
kind of mentality. And questionyou can question of course, but
trust a little bit more. Andthen to Daniel's Point, also, be
(08:16):
more if you're unsure, be moreproactive, reach out, they're in
a spot where they're willing tohelp shape what this looks like
all
Dori Gonzalez-Acevedo (08:25):
about collaboration. Yeah, right. And
that was also a theme throughoutthe last two days, right? There
are lots of ways to collaborateacross a broad area of whether
you're not your sponsor, or yourregulatory body or your vendor
or your contractor or yourwhatever. There's lots of
different ways to contribute.
Practically, right? And, again,it's about education. Did anyone
(08:51):
ever know that? Probably not.
But those are all available toyou on the FDA website. And you
can sign up. It's there. It'sthere. So I'm really hopeful in
that. As we've seen the trend ingeneral, to more direct to
patient influence that we have,we all have collectively as a
(09:15):
society, right to take a moreVocal Point. Daniel also raised
and brilliantly pointed out, thevoice of the patient is there
everywhere, right? Like so.
Advocacy groups, nonprofitgroups, like all of that stuff,
like you people have passionsabout things and want to see
their loved ones get the helpthat they need. And so it's a
(09:38):
very personal thing. And nowwe're seeing that more and more
because it's direct to patientand we're cutting out a lot of
that bureaucracy. And that'sexciting.
Jason Secola (09:56):
Yeah, it is. And that was also kind of
illustrated through couple ofpresentations that we saw that I
think were very interesting aswell. Now, as far as the crowd
of folks that you saw here atthe conference, I was kind of
excited to see that it was agood mix of age. There was a lot
of younger folks out here. Therewas a couple of companies, I
(10:19):
think there were local inIrvine, those types of places
that that good presence acrosstheir organization also. So it
wasn't just two folks fromquality or one person from it.
They came with five, six people,and those five, six people
represented from uppermanagement level down to newer
people and across quality, itengineering, etc. And they were
(10:42):
all in the room together, whichI think is a promising,
Dori Gonzalez-Acevedo (10:45):
I think it's it speaks to also what
we've seen within our consultingpractice that PowerSeller X is
wanting more team basedtraining. Because when you have
the collective conversation witheveryone together and hearing
the same message, it's a loteasier to influence change, or
(11:05):
do that organizational changethat you need to do so using
things like connects as avehicle to do training, or
bringing us in to do teamtraining right across the silos
that have been built up. reallyhelp. We know this from lots of
Organizational Behavior Therapy,right, like that works.
Jason Secola (11:26):
Well, you heard everybody met. You know,
everybody kind of chuckled whensomebody I think it was
yesterday said, you know, thefriction, that concern or that
maybe that was today. Butsomebody said, the friction that
can come as a result of tryingto get organizational change to
take place between teams.
Everybody kind of chuckled. Ithink everybody knew what that's
like they've all experienced it.
But to your point, bringingeverybody out to something like
(11:50):
this, keeping everybody engagedin the process with better and
higher levels of understandingremoves a lot of those friction
barriers, which then makeseverybody's lives easier,
better, less stressful heard mesay lots
Dori Gonzalez-Acevedo (12:02):
of times, right? It's getting comfortable
with the uncomfortable, right,we have to get through that
friction in order to makesomething new and different. And
part of the innovation processis doing that, right, because we
have to fail a lot of times inorder to get to the successful
one. So that's part of thejourney.
Jason Secola (12:20):
Speaking of new and different now, I think this
is just because this issomething we've talked around
this a little bit so far. Butwe've heard this now at multiple
conferences, from multiplepeople. Ai, quote, unquote, new.
This is new, what new things dowe need to do to validate this?
(12:42):
How are we going to validatethis? That line of questioning
or that line of thought? Howwould you go about addressing
that?
Dori Gonzalez-Acevedo (12:50):
Yeah, I think then several panel members
are also spoke to this today.
This is not new. Yet again,we're this, this is not new.
This is a methodology of how wedo verification, validation,
design, control points. This isshould not be put in a silo in a
(13:11):
different box. And you need awhole nother set of policies and
procedures in order to govern itrepeating mistakes to repeat
those mistakes. We want to takegood practices. Now your
organization may not have goodpractices right now. And that's
a different topic.
Jason Secola (13:28):
Yes, that's a foundational issue. Yeah. So
Dori Gonzalez-Acevedo (13:32):
it's a great opportunity to look at
what are you doing. And so,again, that's a lot of what
we've spent most of this yeardoing with organizations,
looking at what they've done,how they can do better what they
need to shed what they need to,to, to double down or modify,
right? One of the things I saidin the panel today was, you
know, kind of, for every one newquote, unquote, SOP you need,
(13:56):
you can need to retract two ofthem. We don't need to
constantly be adding two, Ithink that that's an old mindset
sort of way of what we just needto add to we had an audit and we
have these set five findings.
And so we're going to add a newSOP for each of the findings.
That's not good organizationaleffectiveness. Well, it's
Jason Secola (14:17):
not sustainable anymore, either the rapid pace
of change and innovation and newtechnologies and processes, et
cetera, et cetera. You can't dothat anymore. It's not
sustainable.
Dori Gonzalez-Acevedo (14:29):
So you know, adopting a mindset where
piloting is kind of a normal wayof being how is this going to
work for this particular eitherproduct or process that we're
going to start documentingloosely? Like, what are you
going to do and what's theimportant parts of that what is
really the critical qualityattributes that you're going to
(14:50):
document and then move forward?
Right, but, but putting thingsinto yet again, another
hierarchical structure with alot of SOPs in place? policies
and work instructions, whateveryou'd like to call them, is not
going to get to the root causeand understanding of what it is
What are we doing? Right? Whatis that algorithm actually
solving? Is it just solving abusiness process? issue? Is it
(15:13):
helping you actually getinformation that you can make
meaning from for a medicaldiagnostic? Or is it just good
business practice, because weall need to adopt these things
for our businesses today?
Jason Secola (15:33):
Yeah. And understanding that and
understanding that in ourindustry in certain contexts,
where I think Stephen Cook hadmentioned today, also, he was
looking at some generalsomething that the EU had put
out, right, and it was theydeemed a high risk system had to
deal with something related toHR and who got bonuses or
terminations, or those types ofthings. And a low risk was a
(15:56):
chatbot. The good point for ourindustry chat bots are used to
interact with physicianssometimes and recommendations
around prescribing and dosages,et cetera, et cetera. So it but
it is making sure that you'rereviewing with a critical eye
properly use cases, what's beingused for how is it applicable?
(16:16):
And then what does that mean?
Dori Gonzalez-Acevedo (16:18):
What does what do you need to do then? And
organizations should beexperimenting with all of this,
right, and making somecollective effort to understand
and refine their own business?
Processes? what's what, whereand can you insert that? You
know, what type of bottlenecksdo you have in your business
process today? And how can wemake that more efficient, we're
(16:40):
all like, have too many choicesto make each and every day, too
many decisions that we have tomake. And we need to be able to
utilize this technology in waysthat make us I think Daniel
called it like super, superthinkers or something like that.
You know, like, I'm only as goodas the information that I can
retain at this moment. And Iknow Grace likes to call it now
(17:03):
the C brain, you get to acertain level in your
organization, it seems that youfilter out information. But that
is true. I do filter out moreinformation that I do today than
I did two years ago. Because Ihave now a team that also can do
things for me. I don't need tohold all that information in my
head. But oh, what bait like ifwe had a bot that actually holds
(17:24):
all that information? And then Ican ask the bot from time to
time? What is that informationthat I need? Yeah, right,
because it's like a parking lot,if you will, right? Because we
can't possibly keep all of thatat the same time?
Jason Secola (17:40):
No, no. And I mean, the time it would take
often to go back through andscrub through all that
information refresh. I mean, youknow how it is you, there's
going to be something you do dayin day out for X amount of time,
six months later, if you haven'tdone it at all, you kind of got
to refresh yourself a little bitand having something that could
accelerate that process.
Dori Gonzalez-Acevedo (18:00):
So we saw this with test automation,
right? We saw quality teams comein when we started, you know,
1015 years ago, starting to realdo test automation, quality
teams would often come in and belike, Well, how do I trust that
test automation to do the testautomation, right? Still, today,
I still have quality peoplestill questioning me on that
(18:22):
sort of stuff, too. But at thesame time, it's the same thing.
Right now, it's morecomplicated, because you have
datasets, and you need to makesure the good data is going in
and the how you're questioningall of that sort of stuff,
right? But the concept is stillthe same. You're using a
technology to leverage somethingthat you couldn't possibly do,
right to manually executesomething. X amount of times
(18:47):
where a test automation canimmediately tell you, you know,
green, go, red stop, and thenmake a person then look at
what's going on. It's the sameconcept, right? So we need to be
able to extrapolate all thelearnings that we've done over
the years and apply that to thesame sort of way. We don't need
(19:10):
new policies and procedures, weneed new ways of thinking about
the technology and where and howcan we use it? My thoughts?
Yeah.
Jason Secola (19:19):
Well, yeah. And I think it just to kind of sum
that up a little bit in thecontext of the conference,
right? It seems like there's alot of folks out there that are
in the same headspace of whatyou just mentioned. And that's
good, that people are hopefullylistening to that. While this is
still in its not infancy from itbeing a thing but it as far as
(19:41):
its platform for being adoptednow. Right. So it's kind of
launching into more utilizationmore broadly now. And people are
paying a little bit moreattention to what thought
leaders are saying about it.
Partly because they don't,they're unsure what to do. Yeah.
Dori Gonzalez-Acevedo (19:57):
And there are things to be concerned
about. Yeah. Not that there'snot things to be concerned
about. But it's again, figuringout where to spend your energy
on those things that are to beconcerned about versus things
that don't matter much. And it'sthe same critical thinking that
we're at, we've asked folks todo for years. And we resulted in
(20:18):
checkboxes and forms and thosesorts of things without a lot of
thought, we want to actually nothave that happen. We want folks
to think about, okay, so for Ido this, what does that mean?
And it's a logical way ofthinking, and how do you teach
that? It's hard, right? I'm notsaying any of this is easy. But
(20:40):
the best way to try to get overthat hump is a collaborative
team approach to all
Jason Secola (20:46):
of it. Yeah, well, and I think one big upside,
also, it goes back to somethingthat we were talking about it
dinner with a younger generationof folks who are just more
naturally accustomed to faster,rapid innovation and a state of
being adaptable to that right.
Having those people now in theworkforce, as this stuff is
(21:06):
becoming more prevalent, Ithink, will help the assuming
the folks that are a little bithigher up are having that
collaborative and willing tolisten and bring people to the
table approach. I think thatthat helps.
Dori Gonzalez-Acevedo (21:21):
It's a you know, it's a great point,
Jason. And I think we should putin the show notes, there was a
great speaker at open textworld, when we were at Las Vegas
two weeks ago, right. Around,remix, I think it was her title
of her book, and I forget thename off the top of my head. But
her keynote was very, veryimportant. And a message I think
(21:44):
that everyone really needs tograpple with. There are
currently five generations inthe workforce today. It has
never happened in our lifetime,at all. And there are five
distinct ways of thinking. Andif we don't figure out how to
communicate with all of thosefive different generations, and
(22:07):
learn from each of them, becausethey all have value of some
level, we're not going to beable to continue to move
forward. And and the amount ofoverlap here is a pretty
significant period of time, andalso at the most complicated
technological time. Which I findfascinating. Yeah. So it's a
(22:29):
good book. And I think we shouldput that out for folks to take a
listen to y'all make
Jason Secola (22:34):
sure we get that added in here. I got to note it.
So we'll get it in there. Soyeah, well, I mean, that's day
and a half of effectively ofwhat we've done so far. another
full day tomorrow is it lookslike there's a lot of good stuff
on on the the agenda. So we'llmake sure we get that out there
in some capacity for people tokind of hear the highlights on
(22:54):
that final day. But any closingthoughts on the conference?
Before we get into some of theseother other questions, I think
the
Dori Gonzalez-Acevedo (23:01):
weather in LA, it's been a nice 70
degrees, and I feel refreshed.
Jason Secola (23:06):
Yeah, it's been pretty fantastic. And it makes
for a nice, easy walk from here,then down to dinner, and all
that kind of stuff. So allright, so let's jump into a
couple of the questions that wegot online. So this is a little
bit of a two part question ortwo questions that somebody
(23:27):
commonly hears from theircustomers, from students. So
let's start with the first onearound risk based CSA. So what
does it take to get there iftheir traditional CSV approach?
And if they struggle to change?
So traditional mindsets, changemanagement, organizational
change? All of that?
Dori Gonzalez-Acevedo (23:48):
Yeah.
Well, you said it all rightthere, right. This is the
hardest part, right? And so whenwe come in to work with a
company, what some might callthe soft skills, or the the, the
warm and fuzzy things that makepeople kind of like, why is she
doing this? Why do we have to dothis exercise or? I get lots of
(24:11):
that, because the most importantthing is to get people out of
there. What they always did, youhave to break that down. And so
for each individual, there's adifferent thing that goes on for
them. So sometimes, I've heardfolks say, Well, I've gotten
(24:33):
slammed for years from internalauditors. I can't ever get
outside and because I have to dowhat they say because it's an
internal auditor is constantlysaying that this isn't this and
this are not sufficient. Sothere's almost a trauma effect
that's going on. Right. Yeah.
(24:56):
And so yeah, it's a mental kind.
of slowing that down and lettingthem feel heard and understood
of why they did what they did,or, or how things evolved to the
way in which an organization hasdone something. Now, there's a
point in which that how that wasdone is no longer serving the
(25:18):
organization anymore. But inorder to get over that hump, you
kind of have to break it downindividually to have it re
looked at from a teamperspective. So new approach to
what is our new vision together,right, rather than the
(25:39):
individual mindset, but now to ateam collective mindset of,
okay, so if our future lookslike this, and we want it to
have monthly releases, and atthe end of the day, when we are
audited, we want to be able to,to know who to go to, as simple
as that. It could be that we areable to transform how they
(26:05):
collect that information, andmaybe put together a matrix of
who to ask, right of where thedata is, rather than an overly
complicated system in order toget there. But it's really
breaking down those teams andhaving a new vision or a new
mission of how they want to beseen within their organization.
(26:28):
Because it's a perception thing.
Jason Secola (26:33):
Let's focus just on risk based for a second and
understanding of risk, right?
Because this is where this couldbreak down a lot to in what
people define is risk. And whenwe talk about like a risk
assessment, for example, andthen, you know, obviously,
you've seen this more than Ihave, but people apply blanket
high risk categorization tothings, which is not really if
(26:55):
you define what a riskassessment is, you've got to
assess things and just blanketapplying a category to
everything that's not doing aproper assessment, you're not
understanding risk. How wouldyou kind of help people better
understand where to look forrisk? How to Apply risk and what
that means? Yeah,
Dori Gonzalez-Acevedo (27:16):
so Wow, this is a big one. So a couple
things I'm thinking about, youknow, because we see a couple
different varieties of thiseither. There's a quote unquote,
risk based approach in placewhere it's a simple checkbox
thing. And at the end of thecheckbox, there is no real
decision other than it's abinary decision, it's GXP, non
(27:39):
GXP. So the effect of that is,you've gone through this
assessment, and there's been, Idon't know, 50 questions. But at
the end of the day, there'sstill just a binary result. That
leaves the people doing theassessment. And also that
system, kind of, well, what didit matter that I answered all
(28:04):
those 50 questions, but those 50questions actually do matter.
And so, and that's often done insilos, so you'll have one group,
create a, an assessment fordefining some things, quote,
unquote, GXP or non GXP, thenyou'll have another team doing a
security risk assessment. Andthen you'll have another team on
(28:27):
procurement side doing a vendorrisk assessment, and none of
them talk to one another. Right.
So now you're you're doingyou're asking questions, but not
in a methodical way in order tomake meaning about oh, and by
the way, did anyone reallyidentify what the intended use
of said system is? It's moreoften than not a summary from
(28:50):
the vendor about what the systemcan do, and not how that
organization is going to usethat information that they're
going to put into a
Jason Secola (29:06):
proper context, the context then shades what the
responses to those questionsneed to be.
Dori Gonzalez-Acevedo (29:13):
So so this is where I know people get
frustrated sometimes with me inthat I ask so many questions.
And I don't give you an astraight answer. Because all of
that information has to bepushed through because I'm only
good for the information that Ihave at that moment in time to
make a decision on. So we're,we're in an information crisis.
(29:39):
Right? And we need to be able tocollect that information, make
meaning of that information, butmake meaningful decisions from
that information. And this iswhere some of the software that
we have lacks the ability to dothat. Well, some of the times
it's that not Not everything wasconnected and done in silos.
(30:04):
Sometimes it's for convenience,we've just made these blanket
statements that this is a GXP.
And therefore that I've done arisk based approach, because I
deemed it GXP. But it doesn'ttake into account ppi, or HIPAA
or anything else, right. And sowe need to be more willing to
(30:25):
have those nuanced contextualconversations. That is also at
the point in time of whichyou've made that decision today.
Now, if things change, andthings will change, for a
variety of reasons, new releasescome out of that software,
you've learned more about thesoftware and you want to use it
(30:46):
in different ways you theregulations may change, and I
need to add stuff in, or likeall of those things. So this
constant re looking at things,rather than what has been kind
of historically done it from aperiodic review perspective is
just say, Has any, you know, amI compliant? Right. And so we've
(31:07):
used compliant, it mightcompliant in a, almost a bad
term. I don't know if that's theright word I'm looking for. But
we've used it in a way to saywe're okay, but we haven't done
our due diligence and reallyasking the questions that we
want or should be asking. Okay,
Jason Secola (31:33):
that is actually perfect for the next part of
this question. It kind of tiesin, right. So it starts with is
QA onboard yet, right? With alot of the stuff that we're
talking about. So there's lotsof talk floating around about
focus on quality, and not justfocus on being compliant or on
compliance? The question isreally well sell me on how to
(31:56):
make that the case?
Dori Gonzalez-Acevedo (31:57):
So it's a great question, because So
technically, from a, from aregulatory perspective, right?
There are regulations out there,those are the law, we as
organizations need to interpretthat law. And when we interpret
(32:19):
that law, we create our policiesand procedures, in order for our
organizations to abide by. Andwhen we don't abide by them,
we're not in compliance, butwe're not in compliance with our
own organization'sinterpretation of the law, or
how we are going to conductbusiness. That's the only meter
(32:43):
when we're talking aboutcompliance. I think that nuance
gets lost a lot. And we've beentalking about that now for
almost consistently, like thelast two years in every
conference that we talked to.
And the FDA will double down onthat time and time again, I
think the shift to understandwhat is quality is what we
(33:07):
collectively are strugglingwith.
I think it's easy to default todoes it impact drug quality,
patient safety, efficacy, right.
And those are easy, quick termsto do. But if we go back to
(33:32):
basic engineering functions,quality by design is and should
be at the forefront ofeverything you do. Historically,
as our industry has CSV industryhas grown, we've kind of
sidestepped that, and have putin a lot of heavy measures to
(33:58):
control after the fact. And sobecause that's easier to manage,
it's, it's for lots of differentreasons. Right. And, and then
hidden under the word ofcompliance. What we're what I
(34:19):
think what we're asking folks todo is actually embed quality
from the very beginning. Andthat takes a different mindset
to do organizationally,culturally, individually, but
everyone at the end of the daywants to feel value. And that is
(34:40):
where we all can contribute tobecause we all do care about
quality. We're just maybetalking about it in different
ways.
Jason Secola (34:53):
Yeah. Well, I think that pretty well sums that
up and we had that I had theconversation with a few People
today really about, you know,what we try to do what we're
aiming to do when we go in andwork with folks. And it is more
of that embedded quality bydesign approach and putting less
of an emphasis on engineeringthings for quote unquote,
compliance engineering thing,designing things for better
(35:15):
quality outcomes, right. Andthat will lead
Dori Gonzalez-Acevedo (35:17):
the way we might not get it right the
first time. And that is okay.
And I think, you know, again,working in this continuous way
of evolving, I don't really wantto use continuous improvement
anyway, it's a continuousevolution of, of in, you know,
bringing in new technology, newways of thinking, is this
(35:38):
working right now? What do weneed to change? And in service
of the quality of whatever we'redoing, will help everyone move
forward?
Jason Secola (35:49):
Well, that's part of what's being engineered,
right, is having that ability tobe more nimble in place rather
than these rigid structures.
Right. So all right, let's moveon to we got two more questions
here. So, one, how would youconvince college age women
passionate about science, thatquality and validation are good
(36:09):
career paths?
Dori Gonzalez-Acevedo (36:14):
So it's an interesting question,
because, you know, as a chemistand hard sciences for, you know,
almost until 32, that's kind ofall I really did. I never even
really understood this functionuntil I was pushed to have to do
(36:34):
it. But I'm hopeful actually,things are changing, because I
think that there's a way inwhich young folks are pushing
the envelope to want tounderstand the why this
(36:55):
generation questions everything.
And they should. And I thinkthat has power to then say,
well, what is quality? Why,what, what is what added value?
Does the validation team do? Howdo you learn about that? Right.
And so I think it's a veryunique time and place to be able
(37:17):
to establish somethingdifferent. It didn't doesn't
exist, like you don't go toschool to become a validation
engineer. Right? That's not adegree, as far as I'm aware that
anyone offers. But there's goodengineering practices. There's
good, you know, and there's alot of more cross functional and
(37:38):
multidisciplinary STEM programstoday than ever before. And so I
think the challenge is to say,how do we how do we do this
differently? And I think forwomen, it's a combination of
what, I don't want to use theword soft skills anymore, but I
(38:04):
don't know a good one right nowoff the top of my head. It's,
you know, there is operationaland organizational ways in which
women can change a conversationthat, at least today, men can't.
And I think young women have thevoice to be able to do that.
(38:28):
More now than ever.
Jason Secola (38:34):
Well, yeah. And there's there's a couple of
things that you touched on, thatwe've talked about a little bit
already. One is we initiallystarted talking about how many
people raise their hand when youask what they did, and everybody
does kind of multiple things.
And as an another thing wetalked about was the younger
generation being more adaptable,being more involved with these
skills, things and throughexposure of let's just say
(38:58):
software development, right?
Somebody that's into computersciences, software development,
well, you get into softwaredevelopment, it's not just about
learning to write code anymore,right? Because now your dev
testers right, you got tounderstand what testing is
testing all that stuff, thequality around that stuff. What
does that mean? And then anotherarea that we talked about was
all the stuff that's embeddednow in consumer devices,
(39:22):
consumer electronics, you canbuild stuff for the App Store,
there are revenue streams thatare available as an independent
person, that learning theseskills can go out and do and
more and more people, women orotherwise are out there learning
these skills at a younger age.
And I think they're gettingexposed to all the facets of
those things as they're lookingat digital revenue streams,
other areas, and yeah, it's goodexposure,
Dori Gonzalez-Acevedo (39:43):
and at the same time, women are still
underfunded across the board.
And while some are in education,doing better at different
things, it's still an uphillbattle, so we need to still
figure out why is to promoteminorities and women in business
in order to continue that path.
(40:06):
So we get more diversity at thetable. So because of more
diverse team, the better crosseverything we have, right? We
know that we have problems in AIin general, with not being able
to look at diverse populationsin facial recognition. Right. So
(40:31):
these are examples of why weneed to bring more to the table
to start, right. And then weneed to be able to listen to
those as well.
Jason Secola (40:44):
All right. In general, though, would you say
you've seen an uptick in womencoming into the industry in
college studying more of thesethings to actively pursuing?
Dori Gonzalez-Acevedo (40:56):
I don't expert in that in partly why I
say that is because I also wentto an all female graduate
program. And that was heavy inscience. So I've always
surrounded myself with morewomen in science. I think that
there's a lot more programs outthere, but I think it's moving
towards how to elevate theirvoices across the board is what
(41:22):
we all need to work on.
Jason Secola (41:24):
All right. We're on to the last question now.
Dori Gonzalez-Acevedo (41:27):
Okay, before
Jason Secola (41:28):
dinner, yes, before Buca di Beppo. Fun
restaurants to say. So what doyou do? Oh, yeah, what do you do
to stay not only current, butalso to stay as an innovator in
the industry?
Dori Gonzalez-Acevedo (41:43):
Great question. Um, so I am a consumer
of lots of different types ofinformation. part of who I am as
a learner, as I'mneurodivergent, I need lots of
different types of informationin lots of different ways. So I
(42:04):
do podcasts listening, I dowebinars listening, I read
books, I scour websites forinformation. We started Women in
CSV, right, as a collective onLinkedIn as a community for for
folks to get together to come toa place where they can ask
(42:24):
questions. There's coming toconferences like this is
important. And I know not everyorganization can afford that or
spend time for that. And so, butat the same time, there's a lot
of information regulatory bodiesput out on their own website,
(42:45):
right. So yes, I jumped in lastminute to do this panel. And I
didn't have to spend a lot oftime finding the information
that I needed in order to prepfor it. Oh, and by the way, I
use AI and it gets some of thatfor me. Yeah. Right. So I
condensed, what would havenormally been, for me, maybe a
(43:07):
week of prep work, to lead apanel into
Jason Secola (43:12):
an hour that we'll call it on the fly. That's
basically what it amounted
Dori Gonzalez-Acevedo (43:15):
to putting together a whole slide
deck you in so but in the amountof information that I was able
to find and consume and learnmore than what I already knew,
and that hour was huge. Yeah,doesn't take a lot of time. It
takes focus time. Right. So partof, you know, other initiatives
(43:38):
that I do I spend one hour aweek doing, you know, my, I'm
gonna get it wrong, because mydyslexia, the H LW F alliance
that I'm part of. I don't spendenormous amounts of time on that
I spend concentrated amounts oftime on educating myself on
different parts of all thesectors so that I'm up to date
(44:01):
on what's out there. Because Ineed to be educated on what's
out there. Because that's what Ido for a living.
Jason Secola (44:08):
People look to you for that. Yeah, absolutely. But
I
Dori Gonzalez-Acevedo (44:11):
don't have to spend an enormous amount
of time on that, right, becausewe have these modes and
mechanisms to be able to do itmuch more efficiently today.
Jason Secola (44:17):
That's that's definitely a big piece of it for
not just going beyond stayingcurrent, Right. but also
understanding where your headneeds to be from an innovation
standpoint, forward thinking.
But there's, you definitely alsohave a base of this empirical
data hands on experience thatyou've aggregated over time,
right? But you've also reflectedon that I'm sure as you've gone
(44:40):
through that, you're not justdoing the day to day tasks,
you're absorbing it thinkingabout it and that's how you do
it better next time. So there'sthere's a couple of parts in
that that I heard in caseanybody, just to kind of
summarize that piece. One isgeneral curiosity, right the
want and need for informationObviously, passion keeps you
interested in wanting to pursuethat information. And then
(45:03):
three, I think there's a pointwhere part of your your want or
anybody's want for thatinformation, you make yourself
then a part of that community,you talk, you ask questions.
Other people that are innovatorsare thought leaders are then
drawn to you, you're drawn tothem, you have this community,
you see these conferences,right? People kind of want to
(45:24):
elevate each other. They want toask what what do you think about
this? What do you think aboutthis, and you learn through that
community of other like mindedpeople, that when you
demonstrate the curiosity, thewant for that information, the
passion, it kind of puts you inthat group with those people to
then and it's
Dori Gonzalez-Acevedo (45:43):
in the in the way in which we want to
advance what we already know.
Right? It's not? I have lots ofclose relationships with folks
that don't agree with how Iapproach things. Right. And
that's equally as important.
Jason Secola (45:59):
The non echo chamber, correct that it's very,
very important to have, I don'twant to say conflicting
viewpoints. But it is importantto look at things and hear
things from all angles. See,Discord is
Dori Gonzalez-Acevedo (46:11):
really important in this whole thing.
So So I often say like, I spewsomething out. And then I say,
What am I missing? You know, andI want you as well as the rest
of the team at puzzles also tochallenge me like can't like
it's done. It's not how I feel Ineed to because I can't see
those gaps. Right. And that'swhy we advocate for a
(46:33):
multidisciplinary team approachto all of this. We can't you
know, and that's the kind of thebeauty in a quote, unquote,
Agile Model, right? You reallywant individual types of folks
from different backgrounds thatcan do a lot of different
things. Because that ultimatelydrives everyone's curiosity,
(46:55):
everyone's thought process, aslong as everyone's wanting to
innovate.
Jason Secola (46:59):
And it's protection against gaps also,
right. Yeah, absolutely. Allright. Well, look, I think that
about sums it all up. All right,we got a good 47 and a half
minutes recorded. I got an emptystomach. All right. I'm hungry.
All right, very good. Well, thisis great. We'll get it up very
(47:19):
soon. So thanks for taking thetime and chat through all this
stuff. Thanks, Jason. All right.
Talk to you next time.
Okay, so welcome to another episode of software
quality today. I'm Jason Secola.
I'm going to be taking overhosting duties for this episode
anyways, and I'm with
Dori Gonzalez-Acevedo (00:10):
Dori Gonzalez Acevedo your usual
host.
Jason Secola (00:12):
Yes. So what we're going to do today is we're
actually out in the Kanex,medical device validation
University in Anaheim, rightoutside of the gates of the
Magic Kingdom.
Dori Gonzalez-Acevedo (00:25):
Yes, we are. We're just literally a walk
away. Yeah.
Jason Secola (00:28):
So we thought we'd do a little bit of a conference
recap, while we're here. And wealso a couple of weeks ago, put
out for a request for somequestions on LinkedIn for some
folks that we thought maybe youcould spend some time going
over. So we've got all thatstuff queued up. But first of
all, since we're here, sincewe're through two days now of
(00:50):
the conference, I thought maybewe'd kind of dive into what
people are talking about whatwe're hearing about, I know you
let a sort of an impromptu panelthis morning to help adjust for
some folks not showing up thatwe're supposed to be presenting.
You did a solo presentation, butwe also had a chance to sit in
on on a bunch of panels andother presentations as well. So
(01:11):
as there's been pretty common, Ithink at a lot of recent shows
and webinars, AI seems to be avery prevalent theme. A lot of
typical stuff that we see aroundmed device, software's medical
device, validation applications,software, quality applications,
and then broader applications,right. But what do you want to
start with? What were your somesome of your key takeaways?
Dori Gonzalez-Acevedo (01:32):
Well, first, this was a, the first
medical device university thatcan access done solely targeted
at the medical device community.
So for context for everyone,this was, I'm not sure numbers
wise, but it was a small,intimate sort of grouping that's
and folks that actually areproducing medical devices, or
(01:54):
software as a medical device,which is a different audience
than than we've had in the past.
Connect. So I think that that'sreally important. When we kind
of did a show of hands of howmany quality people were in the
room, hands went up galore. Andat the same time, when I asked,
you know, how many engineeringfolks are in the room, a lot of
(02:16):
folks also raise their hands.
Right. So I think it was aunique mix of folks, because a
lot of them play multiple rolesor having to do multiple roles
within their organizations,because they're small, some
small to midsize companies. Butat the same time, we're wearing
lots of hats
Jason Secola (02:37):
seems to be more and more commonplace, regardless
of size of company. Yeah, itseems like the expectation at a
lot of these organizations ismulti skill, multifaceted
hybrid, and it's kind of thenature of how it kind of has to
be in a way. Right.
Dori Gonzalez-Acevedo (02:52):
Yeah. So I think that's important just
for everyone to kind of know.
And then we had, you know, ourpanels in our speakers are from
a variety of different companiesthat are actually doing medical
device manufacturing. And DanielWalter from the FDA was with us,
which was great to be able tokind of give that perspective,
(03:13):
from a regulatory bodyexpectation, which has been
fabulous.
Jason Secola (03:20):
It is it's been really good to see. You know,
obviously, Cisco has been at alot of stuff, Daniel, I've seen
it a lot of stuff, and theirwillingness and want to get out
there and sort of let thecommunity know that they're open
to discussion. They're open toquestions, they're open, be
brought in proactively, theywant to drive innovation, and
(03:42):
not be seen as stiflinginnovation. And their regular
presence at these things, Ithink, kind of illustrates that.
Whether or not you know, peopleare embracing them on that. I
think that varies, but yeah,
Dori Gonzalez-Aceve (03:53):
consistent.
All right, their message isreach out proactively. And in
the Met. Again, this is slightlydifferent than the typical, you
know, connects when we have ageneral validation University
conversation where we're havinga broader mix of drug product,
as well as medical device. Andalso, in my experience, a lot of
medical device companies tend tobe a little bit more forward
(04:16):
thinking in their softwarequality, about how to do that,
because they've had to do thatfor medical device that they
have. And so whether or not it'sa standalone device that has no
software embedded or is that hassome software, but the way they
think about this is more of asoftware engineering mindset.
(04:37):
Then some of the other broaderorganizations that don't
necessarily have that as a corecompetency yet, I say yet
because a lot of what we'reseeing right is more compound or
complex products that are beingbrought up to market right and
that Then the necessity to beable to do this more
(05:02):
collaboratively is in everyone'sbest interest. But you're right.
AI machine learning is a hottopic today. And what I
illustrated in the panel todaywas, again, the first algorithm
was approved in 1995. Yeah.
(05:24):
Right. And there was a very,very slow progression in
algorithms since that time. Butthis has been part of what has
been happening for many years.
And so now, as more broad,broader and more mainstream, the
topic is, right, we're seeingthat more and more people have
(05:46):
to be educated on it andunderstand the role that it
plays, whether it's a part of aproduct or not, you know, what
is the intended use of usingsome of these for my use case
perspective? And being reallyclear about that? And I think
that that we saw that strugglebefore. And I think we're going
(06:07):
to have to double down on how dowe make that better this time
around?
Jason Secola (06:14):
Yeah, I mean, it's a little bit clearer in the
questions people are askingsometimes what they're posing.
And it was also, I don't know ifit was surprising, because it
was a little bit surprising tome when you brought up some of
the stats last night when wewere at dinner about all the
algorithms and other things thathave already been approved by
the FDA. And the number was alot higher than I would have
(06:34):
thought. But that was clear inthe room. Also, when you asked
that I think people thoughtnothing was approved this year,
up to January 2023, or whateverthat date was zero.
Dori Gonzalez-Acevedo (06:44):
And it's not even close. Yeah. Right. And
so this is partly education. Andthe FDA is, you know, been so
transparent through this wholeprocess. It's just whether or
not folks had knew that thatinformation was available,
right? And maybe now becausewe're all talking about it,
we're going to take the nextstep to educate within our own
(07:05):
organizations, and then how todo this and how to apply it.
Where does it apply? Where doesit not apply? And to not my
hope, is that we don't overengineer yet again, and go down
a path that brought us to thisoverburden validation process of
CSV that we have today, in manyorganizations, and we learn a
(07:27):
lesson. And we actually disruptat organization levels to do
something different.
Jason Secola (07:34):
Yeah, well, that kind of goes in line with what
in one of the panels thathappened today, there were some
there's a little bit of back andforth between the consultant
side and the customer side andknowledge gaps and needing to
make decisions. So I think, toyour point, a lot of that stuff
has to do with people need to bemore comfortable. I think
(07:56):
leveraging people on theconsultancy side who have
dedicated expertise in thisstuff, or who are more forward
thinking, trust their judgment,don't get that like heels dug in
kind of mentality. And questionyou can question of course, but
trust a little bit more. Andthen to Daniel's Point, also, be
(08:16):
more if you're unsure, be moreproactive, reach out, they're in
a spot where they're willing tohelp shape what this looks like
all
Dori Gonzalez-Acevedo (08:25):
about collaboration. Yeah, right. And
that was also a theme throughoutthe last two days, right? There
are lots of ways to collaborateacross a broad area of whether
you're not your sponsor, or yourregulatory body or your vendor
or your contractor or yourwhatever. There's lots of
different ways to contribute.
Practically, right? And, again,it's about education. Did anyone
(08:51):
ever know that? Probably not.
But those are all available toyou on the FDA website. And you
can sign up. It's there. It'sthere. So I'm really hopeful in
that. As we've seen the trend ingeneral, to more direct to
patient influence that we have,we all have collectively as a
(09:15):
society, right to take a moreVocal Point. Daniel also raised
and brilliantly pointed out, thevoice of the patient is there
everywhere, right? Like so.
Advocacy groups, nonprofitgroups, like all of that stuff,
like you people have passionsabout things and want to see
their loved ones get the helpthat they need. And so it's a
(09:38):
very personal thing. And nowwe're seeing that more and more
because it's direct to patientand we're cutting out a lot of
that bureaucracy. And that'sexciting.
Jason Secola (09:56):
Yeah, it is. And that was also kind of
illustrated through couple ofpresentations that we saw that I
think were very interesting aswell. Now, as far as the crowd
of folks that you saw here atthe conference, I was kind of
excited to see that it was agood mix of age. There was a lot
of younger folks out here. Therewas a couple of companies, I
(10:19):
think there were local inIrvine, those types of places
that that good presence acrosstheir organization also. So it
wasn't just two folks fromquality or one person from it.
They came with five, six people,and those five, six people
represented from uppermanagement level down to newer
people and across quality, itengineering, etc. And they were
(10:42):
all in the room together, whichI think is a promising,
Dori Gonzalez-Acevedo (10:45):
I think it's it speaks to also what
we've seen within our consultingpractice that PowerSeller X is
wanting more team basedtraining. Because when you have
the collective conversation witheveryone together and hearing
the same message, it's a loteasier to influence change, or
(11:05):
do that organizational changethat you need to do so using
things like connects as avehicle to do training, or
bringing us in to do teamtraining right across the silos
that have been built up. reallyhelp. We know this from lots of
Organizational Behavior Therapy,right, like that works.
Jason Secola (11:26):
Well, you heard everybody met. You know,
everybody kind of chuckled whensomebody I think it was
yesterday said, you know, thefriction, that concern or that
maybe that was today. Butsomebody said, the friction that
can come as a result of tryingto get organizational change to
take place between teams.
Everybody kind of chuckled. Ithink everybody knew what that's
like they've all experienced it.
But to your point, bringingeverybody out to something like
(11:50):
this, keeping everybody engagedin the process with better and
higher levels of understandingremoves a lot of those friction
barriers, which then makeseverybody's lives easier,
better, less stressful heard mesay lots
Dori Gonzalez-Acevedo (12:02):
of times, right? It's getting comfortable
with the uncomfortable, right,we have to get through that
friction in order to makesomething new and different. And
part of the innovation processis doing that, right, because we
have to fail a lot of times inorder to get to the successful
one. So that's part of thejourney.
Jason Secola (12:20):
Speaking of new and different now, I think this
is just because this issomething we've talked around
this a little bit so far. Butwe've heard this now at multiple
conferences, from multiplepeople. Ai, quote, unquote, new.
This is new, what new things dowe need to do to validate this?
(12:42):
How are we going to validatethis? That line of questioning
or that line of thought? Howwould you go about addressing
that?
Dori Gonzalez-Acevedo (12:50):
Yeah, I think then several panel members
are also spoke to this today.
This is not new. Yet again,we're this, this is not new.
This is a methodology of how wedo verification, validation,
design, control points. This isshould not be put in a silo in a
(13:11):
different box. And you need awhole nother set of policies and
procedures in order to govern itrepeating mistakes to repeat
those mistakes. We want to takegood practices. Now your
organization may not have goodpractices right now. And that's
a different topic.
Jason Secola (13:28):
Yes, that's a foundational issue. Yeah. So
Dori Gonzalez-Acevedo (13:32):
it's a great opportunity to look at
what are you doing. And so,again, that's a lot of what
we've spent most of this yeardoing with organizations,
looking at what they've done,how they can do better what they
need to shed what they need to,to, to double down or modify,
right? One of the things I saidin the panel today was, you
know, kind of, for every one newquote, unquote, SOP you need,
(13:56):
you can need to retract two ofthem. We don't need to
constantly be adding two, Ithink that that's an old mindset
sort of way of what we just needto add to we had an audit and we
have these set five findings.
And so we're going to add a newSOP for each of the findings.
That's not good organizationaleffectiveness. Well, it's
Jason Secola (14:17):
not sustainable anymore, either the rapid pace
of change and innovation and newtechnologies and processes, et
cetera, et cetera. You can't dothat anymore. It's not
sustainable.
Dori Gonzalez-Acevedo (14:29):
So you know, adopting a mindset where
piloting is kind of a normal wayof being how is this going to
work for this particular eitherproduct or process that we're
going to start documentingloosely? Like, what are you
going to do and what's theimportant parts of that what is
really the critical qualityattributes that you're going to
(14:50):
document and then move forward?
Right, but, but putting thingsinto yet again, another
hierarchical structure with alot of SOPs in place? policies
and work instructions, whateveryou'd like to call them, is not
going to get to the root causeand understanding of what it is
What are we doing? Right? Whatis that algorithm actually
solving? Is it just solving abusiness process? issue? Is it
(15:13):
helping you actually getinformation that you can make
meaning from for a medicaldiagnostic? Or is it just good
business practice, because weall need to adopt these things
for our businesses today?
Jason Secola (15:33):
Yeah. And understanding that and
understanding that in ourindustry in certain contexts,
where I think Stephen Cook hadmentioned today, also, he was
looking at some generalsomething that the EU had put
out, right, and it was theydeemed a high risk system had to
deal with something related toHR and who got bonuses or
terminations, or those types ofthings. And a low risk was a
(15:56):
chatbot. The good point for ourindustry chat bots are used to
interact with physicianssometimes and recommendations
around prescribing and dosages,et cetera, et cetera. So it but
it is making sure that you'rereviewing with a critical eye
properly use cases, what's beingused for how is it applicable?
(16:16):
And then what does that mean?
Dori Gonzalez-Acevedo (16:18):
What does what do you need to do then? And
organizations should beexperimenting with all of this,
right, and making somecollective effort to understand
and refine their own business?
Processes? what's what, whereand can you insert that? You
know, what type of bottlenecksdo you have in your business
process today? And how can wemake that more efficient, we're
(16:40):
all like, have too many choicesto make each and every day, too
many decisions that we have tomake. And we need to be able to
utilize this technology in waysthat make us I think Daniel
called it like super, superthinkers or something like that.
You know, like, I'm only as goodas the information that I can
retain at this moment. And Iknow Grace likes to call it now
(17:03):
the C brain, you get to acertain level in your
organization, it seems that youfilter out information. But that
is true. I do filter out moreinformation that I do today than
I did two years ago. Because Ihave now a team that also can do
things for me. I don't need tohold all that information in my
head. But oh, what bait like ifwe had a bot that actually holds
(17:24):
all that information? And then Ican ask the bot from time to
time? What is that informationthat I need? Yeah, right,
because it's like a parking lot,if you will, right? Because we
can't possibly keep all of thatat the same time?
Jason Secola (17:40):
No, no. And I mean, the time it would take
often to go back through andscrub through all that
information refresh. I mean, youknow how it is you, there's
going to be something you do dayin day out for X amount of time,
six months later, if you haven'tdone it at all, you kind of got
to refresh yourself a little bitand having something that could
accelerate that process.
Dori Gonzalez-Acevedo (18:00):
So we saw this with test automation,
right? We saw quality teams comein when we started, you know,
1015 years ago, starting to realdo test automation, quality
teams would often come in and belike, Well, how do I trust that
test automation to do the testautomation, right? Still, today,
I still have quality peoplestill questioning me on that
(18:22):
sort of stuff, too. But at thesame time, it's the same thing.
Right now, it's morecomplicated, because you have
datasets, and you need to makesure the good data is going in
and the how you're questioningall of that sort of stuff,
right? But the concept is stillthe same. You're using a
technology to leverage somethingthat you couldn't possibly do,
right to manually executesomething. X amount of times
(18:47):
where a test automation canimmediately tell you, you know,
green, go, red stop, and thenmake a person then look at
what's going on. It's the sameconcept, right? So we need to be
able to extrapolate all thelearnings that we've done over
the years and apply that to thesame sort of way. We don't need
(19:10):
new policies and procedures, weneed new ways of thinking about
the technology and where and howcan we use it? My thoughts?
Yeah.
Jason Secola (19:19):
Well, yeah. And I think it just to kind of sum
that up a little bit in thecontext of the conference,
right? It seems like there's alot of folks out there that are
in the same headspace of whatyou just mentioned. And that's
good, that people are hopefullylistening to that. While this is
still in its not infancy from itbeing a thing but it as far as
(19:41):
its platform for being adoptednow. Right. So it's kind of
launching into more utilizationmore broadly now. And people are
paying a little bit moreattention to what thought
leaders are saying about it.
Partly because they don't,they're unsure what to do. Yeah.
Dori Gonzalez-Acevedo (19:57):
And there are things to be concerned
about. Yeah. Not that there'snot things to be concerned
about. But it's again, figuringout where to spend your energy
on those things that are to beconcerned about versus things
that don't matter much. And it'sthe same critical thinking that
we're at, we've asked folks todo for years. And we resulted in
(20:18):
checkboxes and forms and thosesorts of things without a lot of
thought, we want to actually nothave that happen. We want folks
to think about, okay, so for Ido this, what does that mean?
And it's a logical way ofthinking, and how do you teach
that? It's hard, right? I'm notsaying any of this is easy. But
(20:40):
the best way to try to get overthat hump is a collaborative
team approach to all
Jason Secola (20:46):
of it. Yeah, well, and I think one big upside,
also, it goes back to somethingthat we were talking about it
dinner with a younger generationof folks who are just more
naturally accustomed to faster,rapid innovation and a state of
being adaptable to that right.
Having those people now in theworkforce, as this stuff is
(21:06):
becoming more prevalent, Ithink, will help the assuming
the folks that are a little bithigher up are having that
collaborative and willing tolisten and bring people to the
table approach. I think thatthat helps.
Dori Gonzalez-Acevedo (21:21):
It's a you know, it's a great point,
Jason. And I think we should putin the show notes, there was a
great speaker at open textworld, when we were at Las Vegas
two weeks ago, right. Around,remix, I think it was her title
of her book, and I forget thename off the top of my head. But
her keynote was very, veryimportant. And a message I think
(21:44):
that everyone really needs tograpple with. There are
currently five generations inthe workforce today. It has
never happened in our lifetime,at all. And there are five
distinct ways of thinking. Andif we don't figure out how to
communicate with all of thosefive different generations, and
(22:07):
learn from each of them, becausethey all have value of some
level, we're not going to beable to continue to move
forward. And and the amount ofoverlap here is a pretty
significant period of time, andalso at the most complicated
technological time. Which I findfascinating. Yeah. So it's a
(22:29):
good book. And I think we shouldput that out for folks to take a
listen to y'all make
Jason Secola (22:34):
sure we get that added in here. I got to note it.
So we'll get it in there. Soyeah, well, I mean, that's day
and a half of effectively ofwhat we've done so far. another
full day tomorrow is it lookslike there's a lot of good stuff
on on the the agenda. So we'llmake sure we get that out there
in some capacity for people tokind of hear the highlights on
(22:54):
that final day. But any closingthoughts on the conference?
Before we get into some of theseother other questions, I think
the
Dori Gonzalez-Acevedo (23:01):
weather in LA, it's been a nice 70
degrees, and I feel refreshed.
Jason Secola (23:06):
Yeah, it's been pretty fantastic. And it makes
for a nice, easy walk from here,then down to dinner, and all
that kind of stuff. So allright, so let's jump into a
couple of the questions that wegot online. So this is a little
bit of a two part question ortwo questions that somebody
(23:27):
commonly hears from theircustomers, from students. So
let's start with the first onearound risk based CSA. So what
does it take to get there iftheir traditional CSV approach?
And if they struggle to change?
So traditional mindsets, changemanagement, organizational
change? All of that?
Dori Gonzalez-Acevedo (23:48):
Yeah.
Well, you said it all rightthere, right. This is the
hardest part, right? And so whenwe come in to work with a
company, what some might callthe soft skills, or the the, the
warm and fuzzy things that makepeople kind of like, why is she
doing this? Why do we have to dothis exercise or? I get lots of
(24:11):
that, because the most importantthing is to get people out of
there. What they always did, youhave to break that down. And so
for each individual, there's adifferent thing that goes on for
them. So sometimes, I've heardfolks say, Well, I've gotten
(24:33):
slammed for years from internalauditors. I can't ever get
outside and because I have to dowhat they say because it's an
internal auditor is constantlysaying that this isn't this and
this are not sufficient. Sothere's almost a trauma effect
that's going on. Right. Yeah.
(24:56):
And so yeah, it's a mental kind.
of slowing that down and lettingthem feel heard and understood
of why they did what they did,or, or how things evolved to the
way in which an organization hasdone something. Now, there's a
point in which that how that wasdone is no longer serving the
(25:18):
organization anymore. But inorder to get over that hump, you
kind of have to break it downindividually to have it re
looked at from a teamperspective. So new approach to
what is our new vision together,right, rather than the
(25:39):
individual mindset, but now to ateam collective mindset of,
okay, so if our future lookslike this, and we want it to
have monthly releases, and atthe end of the day, when we are
audited, we want to be able to,to know who to go to, as simple
as that. It could be that we areable to transform how they
(26:05):
collect that information, andmaybe put together a matrix of
who to ask, right of where thedata is, rather than an overly
complicated system in order toget there. But it's really
breaking down those teams andhaving a new vision or a new
mission of how they want to beseen within their organization.
(26:28):
Because it's a perception thing.
Jason Secola (26:33):
Let's focus just on risk based for a second and
understanding of risk, right?
Because this is where this couldbreak down a lot to in what
people define is risk. And whenwe talk about like a risk
assessment, for example, andthen, you know, obviously,
you've seen this more than Ihave, but people apply blanket
high risk categorization tothings, which is not really if
(26:55):
you define what a riskassessment is, you've got to
assess things and just blanketapplying a category to
everything that's not doing aproper assessment, you're not
understanding risk. How wouldyou kind of help people better
understand where to look forrisk? How to Apply risk and what
that means? Yeah,
Dori Gonzalez-Acevedo (27:16):
so Wow, this is a big one. So a couple
things I'm thinking about, youknow, because we see a couple
different varieties of thiseither. There's a quote unquote,
risk based approach in placewhere it's a simple checkbox
thing. And at the end of thecheckbox, there is no real
decision other than it's abinary decision, it's GXP, non
(27:39):
GXP. So the effect of that is,you've gone through this
assessment, and there's been, Idon't know, 50 questions. But at
the end of the day, there'sstill just a binary result. That
leaves the people doing theassessment. And also that
system, kind of, well, what didit matter that I answered all
(28:04):
those 50 questions, but those 50questions actually do matter.
And so, and that's often done insilos, so you'll have one group,
create a, an assessment fordefining some things, quote,
unquote, GXP or non GXP, thenyou'll have another team doing a
security risk assessment. Andthen you'll have another team on
(28:27):
procurement side doing a vendorrisk assessment, and none of
them talk to one another. Right.
So now you're you're doingyou're asking questions, but not
in a methodical way in order tomake meaning about oh, and by
the way, did anyone reallyidentify what the intended use
of said system is? It's moreoften than not a summary from
(28:50):
the vendor about what the systemcan do, and not how that
organization is going to usethat information that they're
going to put into a
Jason Secola (29:06):
proper context, the context then shades what the
responses to those questionsneed to be.
Dori Gonzalez-Acevedo (29:13):
So so this is where I know people get
frustrated sometimes with me inthat I ask so many questions.
And I don't give you an astraight answer. Because all of
that information has to bepushed through because I'm only
good for the information that Ihave at that moment in time to
make a decision on. So we're,we're in an information crisis.
(29:39):
Right? And we need to be able tocollect that information, make
meaning of that information, butmake meaningful decisions from
that information. And this iswhere some of the software that
we have lacks the ability to dothat. Well, some of the times
it's that not Not everything wasconnected and done in silos.
(30:04):
Sometimes it's for convenience,we've just made these blanket
statements that this is a GXP.
And therefore that I've done arisk based approach, because I
deemed it GXP. But it doesn'ttake into account ppi, or HIPAA
or anything else, right. And sowe need to be more willing to
(30:25):
have those nuanced contextualconversations. That is also at
the point in time of whichyou've made that decision today.
Now, if things change, andthings will change, for a
variety of reasons, new releasescome out of that software,
you've learned more about thesoftware and you want to use it
(30:46):
in different ways you theregulations may change, and I
need to add stuff in, or likeall of those things. So this
constant re looking at things,rather than what has been kind
of historically done it from aperiodic review perspective is
just say, Has any, you know, amI compliant? Right. And so we've
(31:07):
used compliant, it mightcompliant in a, almost a bad
term. I don't know if that's theright word I'm looking for. But
we've used it in a way to saywe're okay, but we haven't done
our due diligence and reallyasking the questions that we
want or should be asking. Okay,
Jason Secola (31:33):
that is actually perfect for the next part of
this question. It kind of tiesin, right. So it starts with is
QA onboard yet, right? With alot of the stuff that we're
talking about. So there's lotsof talk floating around about
focus on quality, and not justfocus on being compliant or on
compliance? The question isreally well sell me on how to
(31:56):
make that the case?
Dori Gonzalez-Acevedo (31:57):
So it's a great question, because So
technically, from a, from aregulatory perspective, right?
There are regulations out there,those are the law, we as
organizations need to interpretthat law. And when we interpret
(32:19):
that law, we create our policiesand procedures, in order for our
organizations to abide by. Andwhen we don't abide by them,
we're not in compliance, butwe're not in compliance with our
own organization'sinterpretation of the law, or
how we are going to conductbusiness. That's the only meter
(32:43):
when we're talking aboutcompliance. I think that nuance
gets lost a lot. And we've beentalking about that now for
almost consistently, like thelast two years in every
conference that we talked to.
And the FDA will double down onthat time and time again, I
think the shift to understandwhat is quality is what we
(33:07):
collectively are strugglingwith.
I think it's easy to default todoes it impact drug quality,
patient safety, efficacy, right.
And those are easy, quick termsto do. But if we go back to
(33:32):
basic engineering functions,quality by design is and should
be at the forefront ofeverything you do. Historically,
as our industry has CSV industryhas grown, we've kind of
sidestepped that, and have putin a lot of heavy measures to
(33:58):
control after the fact. And sobecause that's easier to manage,
it's, it's for lots of differentreasons. Right. And, and then
hidden under the word ofcompliance. What we're what I
(34:19):
think what we're asking folks todo is actually embed quality
from the very beginning. Andthat takes a different mindset
to do organizationally,culturally, individually, but
everyone at the end of the daywants to feel value. And that is
(34:40):
where we all can contribute tobecause we all do care about
quality. We're just maybetalking about it in different
ways.
Jason Secola (34:53):
Yeah. Well, I think that pretty well sums that
up and we had that I had theconversation with a few People
today really about, you know,what we try to do what we're
aiming to do when we go in andwork with folks. And it is more
of that embedded quality bydesign approach and putting less
of an emphasis on engineeringthings for quote unquote,
compliance engineering thing,designing things for better
(35:15):
quality outcomes, right. Andthat will lead
Dori Gonzalez-Acevedo (35:17):
the way we might not get it right the
first time. And that is okay.
And I think, you know, again,working in this continuous way
of evolving, I don't really wantto use continuous improvement
anyway, it's a continuousevolution of, of in, you know,
bringing in new technology, newways of thinking, is this
(35:38):
working right now? What do weneed to change? And in service
of the quality of whatever we'redoing, will help everyone move
forward?
Jason Secola (35:49):
Well, that's part of what's being engineered,
right, is having that ability tobe more nimble in place rather
than these rigid structures.
Right. So all right, let's moveon to we got two more questions
here. So, one, how would youconvince college age women
passionate about science, thatquality and validation are good
(36:09):
career paths?
Dori Gonzalez-Acevedo (36:14):
So it's an interesting question,
because, you know, as a chemistand hard sciences for, you know,
almost until 32, that's kind ofall I really did. I never even
really understood this functionuntil I was pushed to have to do
(36:34):
it. But I'm hopeful actually,things are changing, because I
think that there's a way inwhich young folks are pushing
the envelope to want tounderstand the why this
(36:55):
generation questions everything.
And they should. And I thinkthat has power to then say,
well, what is quality? Why,what, what is what added value?
Does the validation team do? Howdo you learn about that? Right.
And so I think it's a veryunique time and place to be able
(37:17):
to establish somethingdifferent. It didn't doesn't
exist, like you don't go toschool to become a validation
engineer. Right? That's not adegree, as far as I'm aware that
anyone offers. But there's goodengineering practices. There's
good, you know, and there's alot of more cross functional and
(37:38):
multidisciplinary STEM programstoday than ever before. And so I
think the challenge is to say,how do we how do we do this
differently? And I think forwomen, it's a combination of
what, I don't want to use theword soft skills anymore, but I
(38:04):
don't know a good one right nowoff the top of my head. It's,
you know, there is operationaland organizational ways in which
women can change a conversationthat, at least today, men can't.
And I think young women have thevoice to be able to do that.
(38:28):
More now than ever.
Jason Secola (38:34):
Well, yeah. And there's there's a couple of
things that you touched on, thatwe've talked about a little bit
already. One is we initiallystarted talking about how many
people raise their hand when youask what they did, and everybody
does kind of multiple things.
And as an another thing wetalked about was the younger
generation being more adaptable,being more involved with these
skills, things and throughexposure of let's just say
(38:58):
software development, right?
Somebody that's into computersciences, software development,
well, you get into softwaredevelopment, it's not just about
learning to write code anymore,right? Because now your dev
testers right, you got tounderstand what testing is
testing all that stuff, thequality around that stuff. What
does that mean? And then anotherarea that we talked about was
all the stuff that's embeddednow in consumer devices,
(39:22):
consumer electronics, you canbuild stuff for the App Store,
there are revenue streams thatare available as an independent
person, that learning theseskills can go out and do and
more and more people, women orotherwise are out there learning
these skills at a younger age.
And I think they're gettingexposed to all the facets of
those things as they're lookingat digital revenue streams,
other areas, and yeah, it's goodexposure,
Dori Gonzalez-Acevedo (39:43):
and at the same time, women are still
underfunded across the board.
And while some are in education,doing better at different
things, it's still an uphillbattle, so we need to still
figure out why is to promoteminorities and women in business
in order to continue that path.
(40:06):
So we get more diversity at thetable. So because of more
diverse team, the better crosseverything we have, right? We
know that we have problems in AIin general, with not being able
to look at diverse populationsin facial recognition. Right. So
(40:31):
these are examples of why weneed to bring more to the table
to start, right. And then weneed to be able to listen to
those as well.
Jason Secola (40:44):
All right. In general, though, would you say
you've seen an uptick in womencoming into the industry in
college studying more of thesethings to actively pursuing?
Dori Gonzalez-Acevedo (40:56):
I don't expert in that in partly why I
say that is because I also wentto an all female graduate
program. And that was heavy inscience. So I've always
surrounded myself with morewomen in science. I think that
there's a lot more programs outthere, but I think it's moving
towards how to elevate theirvoices across the board is what
(41:22):
we all need to work on.
Jason Secola (41:24):
All right. We're on to the last question now.
Dori Gonzalez-Acevedo (41:27):
Okay, before
Jason Secola (41:28):
dinner, yes, before Buca di Beppo. Fun
restaurants to say. So what doyou do? Oh, yeah, what do you do
to stay not only current, butalso to stay as an innovator in
the industry?
Dori Gonzalez-Acevedo (41:43):
Great question. Um, so I am a consumer
of lots of different types ofinformation. part of who I am as
a learner, as I'mneurodivergent, I need lots of
different types of informationin lots of different ways. So I
(42:04):
do podcasts listening, I dowebinars listening, I read
books, I scour websites forinformation. We started Women in
CSV, right, as a collective onLinkedIn as a community for for
folks to get together to come toa place where they can ask
(42:24):
questions. There's coming toconferences like this is
important. And I know not everyorganization can afford that or
spend time for that. And so, butat the same time, there's a lot
of information regulatory bodiesput out on their own website,
(42:45):
right. So yes, I jumped in lastminute to do this panel. And I
didn't have to spend a lot oftime finding the information
that I needed in order to prepfor it. Oh, and by the way, I
use AI and it gets some of thatfor me. Yeah. Right. So I
condensed, what would havenormally been, for me, maybe a
(43:07):
week of prep work, to lead apanel into
Jason Secola (43:12):
an hour that we'll call it on the fly. That's
basically what it amounted
Dori Gonzalez-Acevedo (43:15):
to putting together a whole slide
deck you in so but in the amountof information that I was able
to find and consume and learnmore than what I already knew,
and that hour was huge. Yeah,doesn't take a lot of time. It
takes focus time. Right. So partof, you know, other initiatives
(43:38):
that I do I spend one hour aweek doing, you know, my, I'm
gonna get it wrong, because mydyslexia, the H LW F alliance
that I'm part of. I don't spendenormous amounts of time on that
I spend concentrated amounts oftime on educating myself on
different parts of all thesectors so that I'm up to date
(44:01):
on what's out there. Because Ineed to be educated on what's
out there. Because that's what Ido for a living.
Jason Secola (44:08):
People look to you for that. Yeah, absolutely. But
I
Dori Gonzalez-Acevedo (44:11):
don't have to spend an enormous amount
of time on that, right, becausewe have these modes and
mechanisms to be able to do itmuch more efficiently today.
Jason Secola (44:17):
That's that's definitely a big piece of it for
not just going beyond stayingcurrent, Right. but also
understanding where your headneeds to be from an innovation
standpoint, forward thinking.
But there's, you definitely alsohave a base of this empirical
data hands on experience thatyou've aggregated over time,
right? But you've also reflectedon that I'm sure as you've gone
(44:40):
through that, you're not justdoing the day to day tasks,
you're absorbing it thinkingabout it and that's how you do
it better next time. So there'sthere's a couple of parts in
that that I heard in caseanybody, just to kind of
summarize that piece. One isgeneral curiosity, right the
want and need for informationObviously, passion keeps you
interested in wanting to pursuethat information. And then
(45:03):
three, I think there's a pointwhere part of your your want or
anybody's want for thatinformation, you make yourself
then a part of that community,you talk, you ask questions.
Other people that are innovatorsare thought leaders are then
drawn to you, you're drawn tothem, you have this community,
you see these conferences,right? People kind of want to
(45:24):
elevate each other. They want toask what what do you think about
this? What do you think aboutthis, and you learn through that
community of other like mindedpeople, that when you
demonstrate the curiosity, thewant for that information, the
passion, it kind of puts you inthat group with those people to
then and it's
Dori Gonzalez-Acevedo (45:43):
in the in the way in which we want to
advance what we already know.
Right? It's not? I have lots ofclose relationships with folks
that don't agree with how Iapproach things. Right. And
that's equally as important.
Jason Secola (45:59):
The non echo chamber, correct that it's very,
very important to have, I don'twant to say conflicting
viewpoints. But it is importantto look at things and hear
things from all angles. See,Discord is
Dori Gonzalez-Acevedo (46:11):
really important in this whole thing.
So So I often say like, I spewsomething out. And then I say,
What am I missing? You know, andI want you as well as the rest
of the team at puzzles also tochallenge me like can't like
it's done. It's not how I feel Ineed to because I can't see
those gaps. Right. And that'swhy we advocate for a
(46:33):
multidisciplinary team approachto all of this. We can't you
know, and that's the kind of thebeauty in a quote, unquote,
Agile Model, right? You reallywant individual types of folks
from different backgrounds thatcan do a lot of different
things. Because that ultimatelydrives everyone's curiosity,
(46:55):
everyone's thought process, aslong as everyone's wanting to
innovate.
Jason Secola (46:59):
And it's protection against gaps also,
right. Yeah, absolutely. Allright. Well, look, I think that
about sums it all up. All right,we got a good 47 and a half
minutes recorded. I got an emptystomach. All right. I'm hungry.
All right, very good. Well, thisis great. We'll get it up very
(47:19):
soon. So thanks for taking thetime and chat through all this
stuff. Thanks, Jason. All right.
Talk to you next time.
Popular Podcasts
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.