Article 19. Algorithmic System Reviews: Substantive vs. Controls Testing

Spoken by a human version of this article.

• Knowing the basics of substantive testing vs. controls testing can help you determine if the review will meet your needs.
• Substantive testing directly identifies errors or unfairness, while controls testing evaluates governance effectiveness. The results/conclusions are different.
• Understanding these differences can also help you anticipate the extent of your team's involvement during the review process.

Links
 This article details a (largely) substantive testing method for accuracy reviews.

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Episode Transcript

(00:00):
This article was publishedin December 2024.
It's titled Algorithmic SystemReviews, Substantive vs.
Controls Testing.
Summary of this isknowing the basics of
Substantive Testing vs.
Controls Testing can helpyou determine if the review
will meet your needs.
Substantive Testing directlyidentifies errors or unfairness

(00:22):
while Controls Testing evaluatesgovernance effectiveness.
The results orconclusions are different.
Understanding thesedifferences can also help
you anticipate the extentof your team's involvement
during the review process.
Here's the full article.
If you are commissioning areview of your algorithmic
system, you ordinarily won'tbe concerned with precisely how

(00:45):
the review will be conducted.
But a high levelunderstanding is important.
It can help you determinehow robust the review will
be and what level of comfortthe result will provide.
It also helps work out upfront what will be needed
from your team for the review.
One aspect to understandis whether the review will

(01:05):
include controls testing,substantive testing or both.
Each of these offers distinctbenefits and limitations.
Sometimes you needjust one of them.
Often, you need or want both.
The key difference lies inthe objective of your review.
If you want to identifyerrors or unfairness in

(01:27):
your algorithmic system,you need a substantive test.
For a direct conclusionabout accuracy or fairness,
there's a link to anotherarticle that details a
largely substantive testingmethod for accuracy reviews.
If you want to assessthe effectiveness of your
governance process, youneed controls testing.

(01:49):
This is often neededto fulfill compliance
obligations, meet contractualexpectations, or demonstrate
adherence to standards.
Controls testing onlywon't directly conclude
on accuracy or fairness.
It can conclude on thecontrols in place to ensure
accuracy or fairness.

(02:10):
This distinction is important.
And just a quick note,there is a way to extend the
controls testing to providea broader conclusion, but
this is not typical, it needsthe right types of controls
and the right level of depthin testing those controls.
On the other hand, substantivetesting only, won't conclude

(02:30):
on the controls in place, itoften won't involve evaluating
processes, which are importantbuilding blocks to understand,
and correct if needed.
It can, so that substantivetesting only can, but
often won't, identify theroot causes of issues.
And that's because withsubstantive testing

(02:51):
only, you're not checkingcontrols or processes.
If you want both, so that's aconclusion about integrity and
about the controls in placeto ensure integrity, you need
a combined testing approach.
That's a review that testscontrols and tests the models
or algorithms and outputs.

(03:11):
Let's look at testing results.
The descriptions and exampleshere are quite high level.
Substantive testing results.
So substantive testingprovides direct evidence by
examining the algorithmicsystem's outputs or results.
For example, in a creditdecisioning algorithm review,
substantive testing mightreveal that 5 percent of

(03:33):
declined applications for aspecific demographic group
were incorrectly assessed.
Controls testing results.
So controls testingevaluates the governance
mechanisms surroundingthe algorithmic system.
In the same credit decisioningcontext, controls testing might
uncover that credit scoringmodels were inconsistently

(03:54):
updated throughout the year.
Potentially leading to incorrectassessments and so the key
differences in those results.
In the substantive testingexample, the results showed
what has actually happened.
Controls testing showedthe potential for something
to happen and to not bepicked up when it does.

(04:16):
The substantive testing isa look back, it does not
consider controls so it usuallycan't be used to determine
what might happen in future.
The controls testing but it canhelp determine how sustainably
a process may operate in future,that is if controls continue
to function in the same way.

(04:37):
So to address theselimitations, The two
approaches can be combined.
So a combined testingapproach looks like this.
Combining controls testingand substantive testing
provides a more robust viewof algorithm integrity.
It covers both the operationalframework, that's the controls,

(04:58):
and actual performance,that's the outputs.
Consider an insurancecompany reviewing its
claims processing algorithm.
Controls testing mightinvolve assessing the data
input validations, the changemanagement procedures for
algorithm updates, and themonitoring systems for detecting
unusual claims decisions.

(05:18):
Substantive testing couldinclude reviewing process
claims to verify accuracy,analyzing the distribution
of claim outcomes acrossdifferent policyholder
groups, and examining casesflagged as potential fraud.
By combining these approaches,the review offers assurance
on both the effectivenessof governance mechanisms.

(05:40):
and the validity ofindividual claim decisions.
You need to know what to expect.
In short, you don't needto know the exact details.
Technical knowledge aboutreviews is not necessary.
If you're commissioning areview that is obviously in
order to all need to know,but if you're commissioning
a review, that technicalknowledge is not necessary.

(06:00):
However, understanding thedifferent approaches can help
you anticipate what the reviewwill cover and ensure it
aligns with your objectives.
That's the end of the article.
Thanks for listening.