October 1, 2024 • 8 mins
Spoken (by a human) version of this article.
Even in discussions among AI governance professionals, there seems to be a silent “gen” before AI.
With rapid progress - or rather prominence – of generative AI capabilities, these have taken centre stage.
Amidst this excitement, we mustn't lose sight of the established algorithms and data-enabled workflows driving core business decisions. These range from simple rules-based systems to complex machine learning models, each playing a crucial role in our operations.
In this episode, we'll examine why we need to keep an eye on established algorithmic systems, and how.
About this podcast
A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.
Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
This article was publishedin October 2024, and it's
titled A Balanced Focus on Newand Established Algorithms.
So the background to thisis that there are lots of
discussions around generative AIgoing on, and even when we speak
about normal AI or algorithmsgenerally, the thinking and
discussion and use cases thatcome up are often in the context
(00:23):
of generative AI, so LLMs andother chatbots and things.
The reality is that there'sa lot more that exists, older
workflows, older uses ofmachine learning, et cetera,
that really do, at the momentanyway, power more of what we
do within financial services interms of just basic processes,
(00:44):
but also things like creditscoring, premium pricing,
claims detection, et cetera.
The intent here is toredirect our focus to.
Not just the new, but alsoestablished algorithms
that really do play abig part in what we do.
So here we go.
Even in discussions amongAI governance professionals,
(01:07):
there seems to be asilent gen before AI.
With rapid progress, or ratherprominence, of generative
AI capabilities, thesehave taken center stage.
Large language models, LLMs,Chachipiti and others, and
broader generative AI, so thingsthat generate videos and images.
(01:27):
are dominating discussions.
They are capturing attentionwith their impressive
capabilities, and rightly so.
However, amidst this excitement,we must not lose sight of the
established algorithms anddata enabled workflows that
have been driving our corebusiness decisions for years.
These range from simple rulesbased systems to complex machine
(01:47):
learning models, each playing acrucial role in our operations.
In this article, we'llexamine why we need to
keep an eye on establishedalgorithmic systems, and how.
The spectrum ofalgorithmic complexity.
So in financial services,we encounter a wide range
of algorithmic systems.
And there's a table in thearticle that represents
(02:07):
a basic outline of thecomplexity spectrum.
In practice, the orderof these items that we're
going to discuss may vary.
Many existing systems willuse several algorithms from
different categories together.
So for example, fraud detectionsystems may combine algorithms
from number two we're goingto speak about and number four
(02:29):
to create a broader system.
Which is then more complex thaneither two or four individually.
Let's just call out the table.
From one to six, thefirst type is simple
rules based or workflow.
In banking we havethings like automatic
transaction categorization.
So, for example, classifyingpurchases as groceries on
(02:50):
credit cards and other things.
Or entertainment basedon merchant codes.
In insurance we may havebasic policy eligibility
checks, so for example, thosethat decline coverage for
provisional license holdersfor high performance vehicles.
The second type of algorithmor algorithmic system we
have is advanced rules based.
(03:11):
In banking, this might be multifactor authentication systems
that use a combination ofrules to verify identity, so
checking location and device.
In insurance, it could be claimstriage systems that route claims
to appropriate departmentsbased on multiple criteria,
like claim type or amount.
The third type isstatistical models.
So in banking, a creditscoring model, perhaps.
(03:34):
In insurance, maybea pricing model.
And sometimes this goeswith number four, which are
machine learning models.
Things like in banking,algorithms that detect
fraudulent transactionsin real time.
Or in insurance, models thathelp identify potentially
fraudulent claims.
So statistical and machinelearning are close together.
Sometimes they aremachine learning.
(03:55):
Sometimes they are simplystatistical, there's not
always a distinction betweenthe two, but they can be.
Then we move on.
Number five is deep learningand neural networks.
So in banking, we mighthave models that predict
future cashflow patterns.
In insurance, we may havemodels that help assess
property damage from thingslike satellite imagery.
And then at the top end ofthe complexity spectrum,
(04:16):
we have generative AI.
And I'm sure there are.
More in between and more abovethis, or there will, ultimately
be more above this, but forpurposes of this discussion,
we've put this as the last item.
So in banking, we might havelarge language models that power
conversational AI for thingslike personalized service.
In insurance, we may have largelanguage models that summarize
(04:37):
product disclosure statements tomake them easier for customers
to read, as we moved along thespectrum, we saw increasingly
complex algorithms that canhandle more sophisticated tasks.
With increasingsophistication, there are new
opportunities and challenges.
But more sophisticated doesnot mean more important.
(04:59):
A potentiallyoverlooked challenge.
It's easy to get caughtup in the Gen AI hype.
While the new technologies grabheadlines, a critical issue
often goes unaddressed, andthat's how established systems
still require significant work.
These established systems haveoften not been subject to the
same level of scrutiny andgovernance that we now expect.
(05:21):
So for instance, a longstanding credit scoring
model might be accurate inpredicting defaults, but lack
fairness in its treatmentof certain customer groups.
The expectations aroundfairness are changing, or a
simple system to calculatethird party commissions might
have undetected inaccuracies.
Maybe not at a macrolevel, but with individual
(05:43):
commissions to individualbrokers for individual loans.
Then we have externalthreats, so bad actors
finding new ways to exploitvulnerabilities, security
vulnerabilities that is.
These all pose seriousreputational risks regardless
of their complexity.
Failures in core systems,whether in terms of fairness,
(06:06):
accuracy or security.
can severely damage trustin our institutions.
Given these challenges, howcan we ensure integrity in both
new and established systems?
Addressing the challenge.
Before we dive intospecifics, it's important
to recognize that focusingon generative AI gives an
incomplete, distorted picture.
(06:29):
The use cases, inputs,processes, outputs, and risks
of established systems areoften very different from
those of newer AI technologies.
So it's useful to keepthese use cases at the
forefront when determiningoverall expectations, when
identifying specific risksand designing policies.
And what we mean here isthat, if we're thinking about
(06:50):
a simple machine learningmodel, let's not focus the
risks on generative AI.
Because generative AI is at theforefront, we can't be thinking
about those use cases whenevaluating our expectations and
risks and policies for simpler,if you like, algorithms.
(07:11):
So with that in mind, hereare a few steps that we can
take to address the challenge.
Four in total.
So the first is holistic.
So ensure that algorithmintegrity efforts cover
all systems, not just thelatest AI technologies.
And that's what we'vebeen talking about in
this article all along.
The second is modernize.
So update all the systems tomeet current expectations.
(07:34):
The third is crossfunctional perspectives.
Involve diverse perspectivesto improve fairness.
And this is becomingmore and more important.
And the last item here,and the last item in
this list, but obviouslynot, it's not a complete
list, is threat modeling.
So regularly assessing howbad actors might exploit new
and established algorithms.
The most important of thoseis definitely making sure
(07:56):
that what we do is holistic.
So we cover all systems,not just the latest AI.
Going forward, maintaining abalance between old and new.
As we navigate the excitingworld of new AI technologies,
we must not neglect the criticalwork needed on our established
systems, including thesimplest rule based algorithms.
(08:18):
It's our responsibilityto ensure that all our
algorithms meet the higheststandards of integrity.
This balanced approach isn'tjust good practice, it's
essential for maintainingtrust, mitigating risks
and staying competitive.
This article was publishedin October 2024, and it's
titled A Balanced Focus on Newand Established Algorithms.
So the background to thisis that there are lots of
discussions around generative AIgoing on, and even when we speak
about normal AI or algorithmsgenerally, the thinking and
discussion and use cases thatcome up are often in the context
(00:23):
of generative AI, so LLMs andother chatbots and things.
The reality is that there'sa lot more that exists, older
workflows, older uses ofmachine learning, et cetera,
that really do, at the momentanyway, power more of what we
do within financial services interms of just basic processes,
(00:44):
but also things like creditscoring, premium pricing,
claims detection, et cetera.
The intent here is toredirect our focus to.
Not just the new, but alsoestablished algorithms
that really do play abig part in what we do.
So here we go.
Even in discussions amongAI governance professionals,
(01:07):
there seems to be asilent gen before AI.
With rapid progress, or ratherprominence, of generative
AI capabilities, thesehave taken center stage.
Large language models, LLMs,Chachipiti and others, and
broader generative AI, so thingsthat generate videos and images.
(01:27):
are dominating discussions.
They are capturing attentionwith their impressive
capabilities, and rightly so.
However, amidst this excitement,we must not lose sight of the
established algorithms anddata enabled workflows that
have been driving our corebusiness decisions for years.
These range from simple rulesbased systems to complex machine
(01:47):
learning models, each playing acrucial role in our operations.
In this article, we'llexamine why we need to
keep an eye on establishedalgorithmic systems, and how.
The spectrum ofalgorithmic complexity.
So in financial services,we encounter a wide range
of algorithmic systems.
And there's a table in thearticle that represents
(02:07):
a basic outline of thecomplexity spectrum.
In practice, the orderof these items that we're
going to discuss may vary.
Many existing systems willuse several algorithms from
different categories together.
So for example, fraud detectionsystems may combine algorithms
from number two we're goingto speak about and number four
(02:29):
to create a broader system.
Which is then more complex thaneither two or four individually.
Let's just call out the table.
From one to six, thefirst type is simple
rules based or workflow.
In banking we havethings like automatic
transaction categorization.
So, for example, classifyingpurchases as groceries on
(02:50):
credit cards and other things.
Or entertainment basedon merchant codes.
In insurance we may havebasic policy eligibility
checks, so for example, thosethat decline coverage for
provisional license holdersfor high performance vehicles.
The second type of algorithmor algorithmic system we
have is advanced rules based.
(03:11):
In banking, this might be multifactor authentication systems
that use a combination ofrules to verify identity, so
checking location and device.
In insurance, it could be claimstriage systems that route claims
to appropriate departmentsbased on multiple criteria,
like claim type or amount.
The third type isstatistical models.
So in banking, a creditscoring model, perhaps.
(03:34):
In insurance, maybea pricing model.
And sometimes this goeswith number four, which are
machine learning models.
Things like in banking,algorithms that detect
fraudulent transactionsin real time.
Or in insurance, models thathelp identify potentially
fraudulent claims.
So statistical and machinelearning are close together.
Sometimes they aremachine learning.
(03:55):
Sometimes they are simplystatistical, there's not
always a distinction betweenthe two, but they can be.
Then we move on.
Number five is deep learningand neural networks.
So in banking, we mighthave models that predict
future cashflow patterns.
In insurance, we may havemodels that help assess
property damage from thingslike satellite imagery.
And then at the top end ofthe complexity spectrum,
(04:16):
we have generative AI.
And I'm sure there are.
More in between and more abovethis, or there will, ultimately
be more above this, but forpurposes of this discussion,
we've put this as the last item.
So in banking, we might havelarge language models that power
conversational AI for thingslike personalized service.
In insurance, we may have largelanguage models that summarize
(04:37):
product disclosure statements tomake them easier for customers
to read, as we moved along thespectrum, we saw increasingly
complex algorithms that canhandle more sophisticated tasks.
With increasingsophistication, there are new
opportunities and challenges.
But more sophisticated doesnot mean more important.
(04:59):
A potentiallyoverlooked challenge.
It's easy to get caughtup in the Gen AI hype.
While the new technologies grabheadlines, a critical issue
often goes unaddressed, andthat's how established systems
still require significant work.
These established systems haveoften not been subject to the
same level of scrutiny andgovernance that we now expect.
(05:21):
So for instance, a longstanding credit scoring
model might be accurate inpredicting defaults, but lack
fairness in its treatmentof certain customer groups.
The expectations aroundfairness are changing, or a
simple system to calculatethird party commissions might
have undetected inaccuracies.
Maybe not at a macrolevel, but with individual
(05:43):
commissions to individualbrokers for individual loans.
Then we have externalthreats, so bad actors
finding new ways to exploitvulnerabilities, security
vulnerabilities that is.
These all pose seriousreputational risks regardless
of their complexity.
Failures in core systems,whether in terms of fairness,
(06:06):
accuracy or security.
can severely damage trustin our institutions.
Given these challenges, howcan we ensure integrity in both
new and established systems?
Addressing the challenge.
Before we dive intospecifics, it's important
to recognize that focusingon generative AI gives an
incomplete, distorted picture.
(06:29):
The use cases, inputs,processes, outputs, and risks
of established systems areoften very different from
those of newer AI technologies.
So it's useful to keepthese use cases at the
forefront when determiningoverall expectations, when
identifying specific risksand designing policies.
And what we mean here isthat, if we're thinking about
(06:50):
a simple machine learningmodel, let's not focus the
risks on generative AI.
Because generative AI is at theforefront, we can't be thinking
about those use cases whenevaluating our expectations and
risks and policies for simpler,if you like, algorithms.
(07:11):
So with that in mind, hereare a few steps that we can
take to address the challenge.
Four in total.
So the first is holistic.
So ensure that algorithmintegrity efforts cover
all systems, not just thelatest AI technologies.
And that's what we'vebeen talking about in
this article all along.
The second is modernize.
So update all the systems tomeet current expectations.
(07:34):
The third is crossfunctional perspectives.
Involve diverse perspectivesto improve fairness.
And this is becomingmore and more important.
And the last item here,and the last item in
this list, but obviouslynot, it's not a complete
list, is threat modeling.
So regularly assessing howbad actors might exploit new
and established algorithms.
The most important of thoseis definitely making sure
(07:56):
that what we do is holistic.
So we cover all systems,not just the latest AI.
Going forward, maintaining abalance between old and new.
As we navigate the excitingworld of new AI technologies,
we must not neglect the criticalwork needed on our established
systems, including thesimplest rule based algorithms.
(08:18):
It's our responsibilityto ensure that all our
algorithms meet the higheststandards of integrity.
This balanced approach isn'tjust good practice, it's
essential for maintainingtrust, mitigating risks
and staying competitive.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy And Charlamagne Tha God!
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.