October 22, 2024 • 9 mins
Spoken (by a human) version of this article.
The integrity of algorithmic systems goes beyond accuracy and fairness.
In Episode 4, we outlined 10 key aspects of algorithm integrity.
Number 5 in that list (not in order of importance) is Security: the algorithmic system needs to be protected from unauthorised access, manipulation and exploitation.
In this episode, we explore one important sub-component of this: deprovisioning user access.
Link from article: U.S. National Coordinator for Critical Infrastructure Security and Resilience (CISA) advisory.
About this podcast
A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.
Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
This article waspublished in October 2024.
It's titled DeprovisioningUser Access to Maintain
Algorithm Integrity.
Security of course is animportant aspect of algorithm
integrity and so some thingsthat we've been seeing
recently around user accessand security related matters
is deprovisioning, whichis about making sure that
(00:24):
terminated employees don'thave access anymore and people
that just don't need accessanymore or haven't been using
their accounts Don't continueto maintain that access.
So this article will explainbriefly why this is important
and then what needs to be done.
So here we go.
The integrity of algorithmicsystems goes beyond core
(00:44):
algorithm accuracy and fairness.
In a previous articlewe outlined key aspects
of algorithm integrity.
Number five in that list,of course not in any order
of importance, is security.
The algorithmic systemneeds to be protected
from unauthorized access,manipulation and exploitation.
In this article, we'llexplore one of the key
(01:04):
subcomponents of this,deprovisioning user access.
Let's start by understandingdeprovisioning.
revoking user access helpsmaintain security and by
extension algorithm integrity.
When users retain access thatthey no longer need or should
no longer have, it can lead tounwanted activity, including
(01:25):
unauthorized transactions,access to confidential data,
IP theft, so for exampleaccess to confidential
models, and exploitation bymalicious actors, things like
denial of service attacks.
We'll focus on two triggersfor deprovisioning,
being termination, forexample of employment,
and dormant accounts.
Other triggers exist, likechanges in job roles, but
(01:48):
termination and dormancy aretypical and fairly risky.
And here are two examplesthat demonstrate this.
So the first isexternal attacks.
The U.
S.
National Coordinator forCritical Infrastructure
Security and Resiliencereleased an advisory recently
and there's a link to thatin the article and we'll put
it in the show notes as well.
That advisory explains thatmalicious actors have targeted
(02:11):
dormant or terminated accounts.
Internal attacks are alittle bit different.
So this is number two.
Second example.
With internal attacks, whatwe often see is that the
access is by an existingemployee using a password left
behind or something similar.
The result is unauthorizedchanges to algorithmic
(02:32):
parameters or data inputs.
It could also beprivacy breaches.
Internal attacks are not oftenexposed for several reasons.
They're also sometimes harderto detect because they come from
within the trusted environment.
And they are internal attacks.
As harsh as that maysound, the activity could
be considered fraudulent.
(02:54):
Get back to the twotriggers and what we can
do to reduce the risk.
The first one is termination.
When users leave, retainingtheir access can lead to
serious security risks thatdirectly reduce integrity.
Here's three of those.
Unauthorized activity, ifaccounts assigned to former
employees are retained, theiraccounts might be used to
(03:15):
initiate unauthorized activity.
A former employee is usuallynot accountable for such actions
after they leave, so anythingdone with their account.
It's the organization'sresponsibility to
remove their access.
Another risk is bypassingsegregation of duties controls.
So where you have makerchecker controls, if a current
(03:35):
employee has access to a formeremployee's account, They could
log in as themselves to initiatethe transaction and then use
the other account to approve it.
Or, for algorithm changecontrol in particular, they
could use one account to makea code change and then use
the other account to migratethat change to production.
Those controls exist toprevent fraud and also errors.
(03:58):
So an errant line of code couldgo unnoticed, resulting in
reduced algorithmic integrity.
Third risk here, or thirdexample of risk is data
breaches or IP theft.
Accounts belonging toformer employees might be
used to view or extractsensitive data or models.
Resulting inconfidentiality breaches.
(04:19):
This can include customerinformation or even
proprietary models.
The solution to thisis reasonably simple.
So here's a coupleof things to do.
Revoke access when an employeeor contractor, including
vendor personnel, leaves.
Disable user access on orbefore the date of termination.
(04:39):
Set up automateddeprovisioning processes
to revoke access on time.
Conduct regular reviews toverify that terminated users no
longer have access to systems.
Now in practice this can takesome effort, but the benefits
generally outweigh the costs.
The second triggerwas dormant accounts.
Accounts that aren'taccessed regularly are risky.
(05:02):
They are prime forexploitation as they're
often not actively monitored.
This could lead to similarrisks to those outlined
for termination earlier.
So for dormant accounts,the solution, first
determine what a reasonableperiod of inactivity is.
For example, if people typicallylog in once a month, perhaps
a threshold of 90 days.
(05:24):
Check account usage regularlyto identify dormant accounts.
So that's accounts that exceedour threshold that we identified
in step one of inactivity.
Deactivate accounts thatare no longer needed.
And consider strengtheningother known dormant accounts.
So for example, if certainservice accounts are used
less frequently, these areusually background accounts
(05:47):
for integrations and things,if those sorts of accounts are
used less frequently but needto be active, then implement
mitigating controls like morecomplex passwords and then
actively monitor their usage.
And finally, considerautomated tools to identify and
deactivate dormant accounts.
So those were the two triggersand some broad risks and broad
(06:09):
solutions to deal with them.
Some additional considerations.
if automated deprovisioning isnot possible or not feasible,
and you use a manual process,exercise extra caution.
and emphasize regular reviews.
For automated deprovisioning,if you can do it, perform
(06:31):
regular checks to make sure theautomation functions correctly.
These things can, and do, break.
When using single sign on, SSOsystems, so this is where you
log in once and don't have toenter passwords again for other
applications, check dormancyat the application level too.
(06:52):
For example, if you useActive Directory or Google
Identity Services forauthentication, single sign
authentication, it is temptingto disable access there,
Active Directory for example,and then assume that all is
well, because that controlsauthentication everywhere.
This can be risky.
applications sometimes allowdirect or non SSO logins.
(07:16):
So this is even thoseapplications that
are primarily SSO.
Sometimes they can allowdirect logins for generic
accounts and for other reasons.
Employees return sometimesto different roles and
then they don't need theaccess they had before.
Reactivating their SSOcredentials automatically gives
them their previous access.
(07:36):
So like reactivatingtheir SSO credentials.
Microsoft Active Directoryaccount will then automatically
give them the application accessif it hasn't been revoked.
It is safer to disable orrevoke access everywhere.
Consider additional measureswhen systems or processes
or behaviors change.
So this includes changeslike these two examples.
(08:00):
First is, if you implementmulti factor authentication
or MFA, and users are forcedto enable MFA when they next
log in, Dormant accountswill remain unprotected.
Dormant accounts mean they'renot logging in and so they
just won't have MFA applied.
So have a plan fordealing with these.
second example here aboutadditional measures for change.
(08:23):
If you automate processesand login activity is more
frequent, perhaps the dormancythreshold should be reduced.
For example, if something liketimesheets were manual and
are now processed through theHR payroll system, employees
may need to log in fortnightlyto complete those timesheets.
If you were using 90 daysas a threshold before, maybe
(08:44):
reduce it to 30 days becausepeople are more active Wrapping
up, proper deprovisioningof user access contributes
to algorithm integrity.
We need to make sure thatonly current and authorized
users have access.
Getting this rightis not difficult.
It does take some effort.
And the effort needsto be sustained.
(09:06):
But without it, werisk losing integrity.
That's the end of the article.
Thanks for listening.
This article waspublished in October 2024.
It's titled DeprovisioningUser Access to Maintain
Algorithm Integrity.
Security of course is animportant aspect of algorithm
integrity and so some thingsthat we've been seeing
recently around user accessand security related matters
is deprovisioning, whichis about making sure that
(00:24):
terminated employees don'thave access anymore and people
that just don't need accessanymore or haven't been using
their accounts Don't continueto maintain that access.
So this article will explainbriefly why this is important
and then what needs to be done.
So here we go.
The integrity of algorithmicsystems goes beyond core
(00:44):
algorithm accuracy and fairness.
In a previous articlewe outlined key aspects
of algorithm integrity.
Number five in that list,of course not in any order
of importance, is security.
The algorithmic systemneeds to be protected
from unauthorized access,manipulation and exploitation.
In this article, we'llexplore one of the key
(01:04):
subcomponents of this,deprovisioning user access.
Let's start by understandingdeprovisioning.
revoking user access helpsmaintain security and by
extension algorithm integrity.
When users retain access thatthey no longer need or should
no longer have, it can lead tounwanted activity, including
(01:25):
unauthorized transactions,access to confidential data,
IP theft, so for exampleaccess to confidential
models, and exploitation bymalicious actors, things like
denial of service attacks.
We'll focus on two triggersfor deprovisioning,
being termination, forexample of employment,
and dormant accounts.
Other triggers exist, likechanges in job roles, but
(01:48):
termination and dormancy aretypical and fairly risky.
And here are two examplesthat demonstrate this.
So the first isexternal attacks.
The U.
S.
National Coordinator forCritical Infrastructure
Security and Resiliencereleased an advisory recently
and there's a link to thatin the article and we'll put
it in the show notes as well.
That advisory explains thatmalicious actors have targeted
(02:11):
dormant or terminated accounts.
Internal attacks are alittle bit different.
So this is number two.
Second example.
With internal attacks, whatwe often see is that the
access is by an existingemployee using a password left
behind or something similar.
The result is unauthorizedchanges to algorithmic
(02:32):
parameters or data inputs.
It could also beprivacy breaches.
Internal attacks are not oftenexposed for several reasons.
They're also sometimes harderto detect because they come from
within the trusted environment.
And they are internal attacks.
As harsh as that maysound, the activity could
be considered fraudulent.
(02:54):
Get back to the twotriggers and what we can
do to reduce the risk.
The first one is termination.
When users leave, retainingtheir access can lead to
serious security risks thatdirectly reduce integrity.
Here's three of those.
Unauthorized activity, ifaccounts assigned to former
employees are retained, theiraccounts might be used to
(03:15):
initiate unauthorized activity.
A former employee is usuallynot accountable for such actions
after they leave, so anythingdone with their account.
It's the organization'sresponsibility to
remove their access.
Another risk is bypassingsegregation of duties controls.
So where you have makerchecker controls, if a current
(03:35):
employee has access to a formeremployee's account, They could
log in as themselves to initiatethe transaction and then use
the other account to approve it.
Or, for algorithm changecontrol in particular, they
could use one account to makea code change and then use
the other account to migratethat change to production.
Those controls exist toprevent fraud and also errors.
(03:58):
So an errant line of code couldgo unnoticed, resulting in
reduced algorithmic integrity.
Third risk here, or thirdexample of risk is data
breaches or IP theft.
Accounts belonging toformer employees might be
used to view or extractsensitive data or models.
Resulting inconfidentiality breaches.
(04:19):
This can include customerinformation or even
proprietary models.
The solution to thisis reasonably simple.
So here's a coupleof things to do.
Revoke access when an employeeor contractor, including
vendor personnel, leaves.
Disable user access on orbefore the date of termination.
(04:39):
Set up automateddeprovisioning processes
to revoke access on time.
Conduct regular reviews toverify that terminated users no
longer have access to systems.
Now in practice this can takesome effort, but the benefits
generally outweigh the costs.
The second triggerwas dormant accounts.
Accounts that aren'taccessed regularly are risky.
(05:02):
They are prime forexploitation as they're
often not actively monitored.
This could lead to similarrisks to those outlined
for termination earlier.
So for dormant accounts,the solution, first
determine what a reasonableperiod of inactivity is.
For example, if people typicallylog in once a month, perhaps
a threshold of 90 days.
(05:24):
Check account usage regularlyto identify dormant accounts.
So that's accounts that exceedour threshold that we identified
in step one of inactivity.
Deactivate accounts thatare no longer needed.
And consider strengtheningother known dormant accounts.
So for example, if certainservice accounts are used
less frequently, these areusually background accounts
(05:47):
for integrations and things,if those sorts of accounts are
used less frequently but needto be active, then implement
mitigating controls like morecomplex passwords and then
actively monitor their usage.
And finally, considerautomated tools to identify and
deactivate dormant accounts.
So those were the two triggersand some broad risks and broad
(06:09):
solutions to deal with them.
Some additional considerations.
if automated deprovisioning isnot possible or not feasible,
and you use a manual process,exercise extra caution.
and emphasize regular reviews.
For automated deprovisioning,if you can do it, perform
(06:31):
regular checks to make sure theautomation functions correctly.
These things can, and do, break.
When using single sign on, SSOsystems, so this is where you
log in once and don't have toenter passwords again for other
applications, check dormancyat the application level too.
(06:52):
For example, if you useActive Directory or Google
Identity Services forauthentication, single sign
authentication, it is temptingto disable access there,
Active Directory for example,and then assume that all is
well, because that controlsauthentication everywhere.
This can be risky.
applications sometimes allowdirect or non SSO logins.
(07:16):
So this is even thoseapplications that
are primarily SSO.
Sometimes they can allowdirect logins for generic
accounts and for other reasons.
Employees return sometimesto different roles and
then they don't need theaccess they had before.
Reactivating their SSOcredentials automatically gives
them their previous access.
(07:36):
So like reactivatingtheir SSO credentials.
Microsoft Active Directoryaccount will then automatically
give them the application accessif it hasn't been revoked.
It is safer to disable orrevoke access everywhere.
Consider additional measureswhen systems or processes
or behaviors change.
So this includes changeslike these two examples.
(08:00):
First is, if you implementmulti factor authentication
or MFA, and users are forcedto enable MFA when they next
log in, Dormant accountswill remain unprotected.
Dormant accounts mean they'renot logging in and so they
just won't have MFA applied.
So have a plan fordealing with these.
second example here aboutadditional measures for change.
(08:23):
If you automate processesand login activity is more
frequent, perhaps the dormancythreshold should be reduced.
For example, if something liketimesheets were manual and
are now processed through theHR payroll system, employees
may need to log in fortnightlyto complete those timesheets.
If you were using 90 daysas a threshold before, maybe
(08:44):
reduce it to 30 days becausepeople are more active Wrapping
up, proper deprovisioningof user access contributes
to algorithm integrity.
We need to make sure thatonly current and authorized
users have access.
Getting this rightis not difficult.
It does take some effort.
And the effort needsto be sustained.
(09:06):
But without it, werisk losing integrity.
That's the end of the article.
Thanks for listening.
Popular Podcasts
United States of Kennedy
United States of Kennedy is a podcast about our cultural fascination with the Kennedy dynasty. Every week, hosts Lyra Smith and George Civeris go into one aspect of the Kennedy story.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.