Article 25. Algorithmic System Integrity: Explainability (Part 2) - Complexity

Spoken by a human version of this article.

TL;DR (TL;DL?)

• Complexity must be actively managed rather than passively accepted.
• Data relevance directly impacts both accuracy and explainability.
• Technical “visibility” techniques can be useful.

To subscribe to the weekly articles: https://riskinsights.com.au/blog#subscribe

About this podcast

A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.

Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Episode Transcript

(00:14):
So this is the second in theseries on explainability.
So this article was titled AlgorithmicSystem Integrity, explainability Part
Two, complexity The Too Long, didn't Read.
Complexity, must Be Actively Managedrather than Passively accepted Data.
Relevance directly impacts bothaccuracy and explainability.

(00:36):
Technical visibilitytechniques can be useful.
So in the last episode, we exploredthe concept of explainability its
importance, and four challenges.
In this one, we explore some solutions tothe first challenge complex algorithms.
So just to recap that challenge complexmodels can operate as black boxes.

(00:57):
Their internal processes are socomplex that even their developers
may struggle to explain howinputs lead to specific outputs.
This poses several problems,and we've listed three.
So the first of those is trustcustomers and other stakeholders
are wary of automated decisionsthat they can't understand.

(01:18):
Second is compliance.
Regulatory frameworks and standardsare now calling for explainability,
especially for decisions with significanthuman impact, and then errors.
When we can't see how our model works,it's harder to spot and fix mistakes.
So onto solutions, then banksand insurers can use several

(01:39):
methods to address this problem.
Here are three sets of commonly usedmethods, each of which work better when
cross-functional teams are involved.
So the first of the threesolutions is simplification.
So we wanna strike a balancebetween accuracy and simplicity.
Simpler models can be easier toexplain, but may not be as accurate

(02:03):
as the more complex models.
There's a diagram in the article,which you can see if you click
through the link to the article, andthat, , basically depicts performance
or accuracy versus explainability.
That article is produced from Zang etal, and it's titled Model Explainability

(02:23):
Versus Model Performance Comparison.
So in general, the more complex modelsthat often produce more accurate
results are harder to explain.
, Simplification can include.
Establishing clear thresholds, sodetermine what an acceptable trade off
between complexity and explainabilityis, and use this to choose the model

(02:44):
that suits implementing validation morecomplex isn't always more accurate.
So don't miss the opportunity to usea model that is easier to explain
and equally or more accurate, andthen developing parallel models.
In some situations, you may want tocreate simpler, interpretable models
that approximate the complex ones.

(03:05):
If you can get them close enough, youmay be able to use them to explain.
Okay, so that's the first solution, apotential solution being simplification.
The second is data selection.
If the data going into the model is notrelevant, it can make it difficult to
determine how decisions are reached.
There are situations where irrelevant datacan result in more accurate predictions.

(03:29):
If the data is not relevant, thenthis accuracy gain might be a
technical improvement, not a real one.
We also need to be careful notto eliminate relevant data, and
this can vary depending on the usecase and the nature of the system.
For both internal data and external data,carefully choose the relevant data and

(03:50):
use data only for its intended purposes.
There's also a discussion abouteliminating protected attributes, and
we'll explore that in a future articlebecause it is quite a long story.
So that's the secondsolution, data selection.
The third are visibility techniques.
These methods work best whencombined with proactive complexity

(04:12):
management from the previous sections.
There are several techniques that wecan use to interpret model processing
and this list of course, as is thelist of solutions, is not exhaustive.
What to use will depend on the use case.
So they are four listed, Thefirst is scenario testing.
We conduct what if analysis to demonstratehow changing inputs affect outcomes

(04:36):
shep?
So SHAPLEY additive explanations,these calculate feature importance
scores and this method is used toexplain the output of machine learning
models by assigning each feature ofvalue for a particular prediction.
Then we have Lyme.
So these are local interpretablemodel agnostic explanations.

(04:57):
Uh, we just call it Lyme.
These create linear explanationsof complex decisions, and this
method is used to explain individualpredictions by approximating the model
locally with an interpretable model.
And then finally we have PDP, so partialdependence plots and ICE, which are
individual conditional expectation plots.

(05:21):
These create population level andinstance specific feature impacts.
These methods are used to visualize howfeatures affect predictions on average.
That's BDP, and for specific instances.
That's ICE.
Now, with all of this said, indetermining which methods to use, we
need to consider regulatory requirements.
So for example, GDPR's,right to explanation and

(05:44):
also operational constraints.
That is the computationalresources that are required.
And in our next article, nextepisode, we'll focus on the second
challenge being complicated processes.
That's the end of this article.
Thanks for listening.